Project

General

Profile

Bug #46800 » ceph-osd.2-excerpt.log

Vitaliy Filippov, 07/31/2020 01:53 PM

 
2020-07-31T16:13:31.388+0300 7f3b6324f700 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f3b6324f700 time 2020-07-31T16:13:31.371632+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x556922a189b7]
2: (()+0x9e1bbf) [0x556922a18bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55692316d918]
4: (BlueStore::_do_alloc_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>, boost::intrusive_ptr<BlueStore::Onode>, BlueStore::WriteContext*)+0x1fb3) [0x556923010383]
5: (BlueStore::_do_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>, unsigned long, unsigned long, ceph::buffer::v15_2_0::list&, unsigned int)+0x338) [0x5569230557d8]
6: (BlueStore::_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, ceph::buffer::v15_2_0::list&, unsigned int)+0xe0) [0x5569230567e0]
7: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x199e) [0x55692305a15e]
8: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x2ad) [0x55692305b42d]
9: (ObjectStore::queue_transaction(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ceph::os::Transaction&&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x87) [0x556922b3e257]
10: (non-virtual thunk to PrimaryLogPG::queue_transaction(ceph::os::Transaction&&, boost::intrusive_ptr<OpRequest>)+0x53) [0x556922ca75d3]
11: (ECBackend::dispatch_recovery_messages(RecoveryMessages&, int)+0xea8) [0x556922ea37d8]
12: (ECBackend::_handle_message(boost::intrusive_ptr<OpRequest>)+0x5a4) [0x556922eb6ec4]
13: (PGBackend::handle_message(boost::intrusive_ptr<OpRequest>)+0x5b) [0x556922ce7cfb]
14: (PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x539) [0x556922c770e9]
15: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x1c0) [0x556922ac98b0]
16: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x66) [0x556922d759b6]
17: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xe8d) [0x556922af6a0d]
18: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x403) [0x5569231d1193]
19: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x5569231d3f94]
20: (()+0x8ea7) [0x7f3b837ddea7]
21: (clone()+0x3f) [0x7f3b8337ddaf]

2020-07-31T16:13:31.400+0300 7f3b6324f700 -1 *** Caught signal (Aborted) **
in thread 7f3b6324f700 thread_name:tp_osd_tp

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f3b837e9140]
2: (gsignal()+0x141) [0x7f3b832bbcb1]
3: (abort()+0x123) [0x7f3b832a5537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x556922a18a12]
5: (()+0x9e1bbf) [0x556922a18bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55692316d918]
7: (BlueStore::_do_alloc_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>, boost::intrusive_ptr<BlueStore::Onode>, BlueStore::WriteContext*)+0x1fb3) [0x556923010383]
8: (BlueStore::_do_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>, unsigned long, unsigned long, ceph::buffer::v15_2_0::list&, unsigned int)+0x338) [0x5569230557d8]
9: (BlueStore::_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, ceph::buffer::v15_2_0::list&, unsigned int)+0xe0) [0x5569230567e0]
10: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x199e) [0x55692305a15e]
11: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x2ad) [0x55692305b42d]
12: (ObjectStore::queue_transaction(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ceph::os::Transaction&&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x87) [0x556922b3e257]
13: (non-virtual thunk to PrimaryLogPG::queue_transaction(ceph::os::Transaction&&, boost::intrusive_ptr<OpRequest>)+0x53) [0x556922ca75d3]
14: (ECBackend::dispatch_recovery_messages(RecoveryMessages&, int)+0xea8) [0x556922ea37d8]
15: (ECBackend::_handle_message(boost::intrusive_ptr<OpRequest>)+0x5a4) [0x556922eb6ec4]
16: (PGBackend::handle_message(boost::intrusive_ptr<OpRequest>)+0x5b) [0x556922ce7cfb]
17: (PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x539) [0x556922c770e9]
18: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x1c0) [0x556922ac98b0]
19: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x66) [0x556922d759b6]
20: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xe8d) [0x556922af6a0d]
21: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x403) [0x5569231d1193]
22: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x5569231d3f94]
23: (()+0x8ea7) [0x7f3b837ddea7]
24: (clone()+0x3f) [0x7f3b8337ddaf]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
2020-07-31T16:13:42.704+0300 7f0b314ffe00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:13:42.704+0300 7f0b314ffe00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1169222
2020-07-31T16:13:42.704+0300 7f0b314ffe00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:13:42.704+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:13:42.704+0300 7f0b314ffe00 1 bdev(0x5605b1e56380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:13:42.704+0300 7f0b314ffe00 1 bdev(0x5605b1e56380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:13:43.020+0300 7f0b314ffe00 1 bdev(0x5605b1e56380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:13:43.264+0300 7f0b314ffe00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:13:43.268+0300 7f0b314ffe00 -1 Falling back to public interface
2020-07-31T16:13:43.284+0300 7f0b314ffe00 0 load: jerasure load: lrc load: isa
2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:13:43.584+0300 7f0b314ffe00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:13:43.584+0300 7f0b314ffe00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:09.952+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:10.156+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:10.156+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:10.156+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:12.020+0300 7f0b314ffe00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f0b314ffe00 time 2020-07-31T16:14:12.018568+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x5605a66e29b7]
2: (()+0x9e1bbf) [0x5605a66e2bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x5605a6e37918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x5605a6cf3c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x5605a6cf6318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x5605a6d0b2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x5605a6d0c2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x5605a6d3b8e1]
9: (OSD::init()+0x61f) [0x5605a67c8daf]
10: (main()+0x3938) [0x5605a6730fb8]
11: (__libc_start_main()+0xea) [0x7f0b31a21cca]
12: (_start()+0x2e) [0x5605a674605e]

2020-07-31T16:14:12.028+0300 7f0b314ffe00 -1 *** Caught signal (Aborted) **
in thread 7f0b314ffe00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f0b31f64140]
2: (gsignal()+0x141) [0x7f0b31a36cb1]
3: (abort()+0x123) [0x7f0b31a20537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x5605a66e2a12]
5: (()+0x9e1bbf) [0x5605a66e2bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x5605a6e37918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x5605a6cf3c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x5605a6cf6318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x5605a6d0b2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x5605a6d0c2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x5605a6d3b8e1]
12: (OSD::init()+0x61f) [0x5605a67c8daf]
13: (main()+0x3938) [0x5605a6730fb8]
14: (__libc_start_main()+0xea) [0x7f0b31a21cca]
15: (_start()+0x2e) [0x5605a674605e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-306> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command assert hook 0x5605b11d6570
-305> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command abort hook 0x5605b11d6570
-304> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command leak_some_memory hook 0x5605b11d6570
-303> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perfcounters_dump hook 0x5605b11d6570
-302> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command 1 hook 0x5605b11d6570
-301> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perf dump hook 0x5605b11d6570
-300> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perfcounters_schema hook 0x5605b11d6570
-299> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perf histogram dump hook 0x5605b11d6570
-298> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command 2 hook 0x5605b11d6570
-297> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perf schema hook 0x5605b11d6570
-296> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perf histogram schema hook 0x5605b11d6570
-295> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command perf reset hook 0x5605b11d6570
-294> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config show hook 0x5605b11d6570
-293> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config help hook 0x5605b11d6570
-292> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config set hook 0x5605b11d6570
-291> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config unset hook 0x5605b11d6570
-290> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config get hook 0x5605b11d6570
-289> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config diff hook 0x5605b11d6570
-288> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command config diff get hook 0x5605b11d6570
-287> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command injectargs hook 0x5605b11d6570
-286> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command log flush hook 0x5605b11d6570
-285> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command log dump hook 0x5605b11d6570
-284> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command log reopen hook 0x5605b11d6570
-283> 2020-07-31T16:13:42.692+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command dump_mempools hook 0x5605b1e4c068
-282> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 10 monclient: get_monmap_and_config
-281> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 10 monclient: build_initial_monmap
-280> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:13:42.704274+0300
created 2020-07-31T16:13:42.704274+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-279> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding auth protocol: cephx
-278> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding auth protocol: cephx
-277> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding auth protocol: cephx
-276> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-275> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-274> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-273> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-272> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-271> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-270> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-269> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-268> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-267> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-266> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-265> 2020-07-31T16:13:42.696+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-264> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-263> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient: init
-262> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding auth protocol: cephx
-261> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding auth protocol: cephx
-260> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding auth protocol: cephx
-259> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: secure
-258> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: crc
-257> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: secure
-256> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: crc
-255> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: secure
-254> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: crc
-253> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: crc
-252> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: secure
-251> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: crc
-250> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: secure
-249> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: crc
-248> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 AuthRegistry(0x7ffda2fcdef0) adding con mode: secure
-247> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-246> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-245> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient: _reopen_session rank -1
-244> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient: _add_conns ranks=[9,8,6,7,4,1,13,3,11,5,0,10,2,12]
-243> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): picked mon.noname-f con 0x5605b1276c00 addr v2:192.168.5.15:3300/0
-242> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): picked mon.noname-e con 0x5605b1277000 addr v2:192.168.5.14:3300/0
-241> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): picked mon.noname-c-legacy con 0x5605b1277400 addr v1:192.168.5.22:6789/0
-240> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): start opening mon connection
-239> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): start opening mon connection
-238> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): _renew_subs
-237> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:18:42.707198+0300
-236> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): get_auth_request con 0x5605b1276c00 auth_method 0
-235> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-234> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): _init_auth method 2
-233> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): handle_auth_reply_more payload 9
-232> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-231> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-230> 2020-07-31T16:13:42.700+0300 7f0b2fca3700 10 monclient(hunting): get_auth_request con 0x5605b1277000 auth_method 0
-229> 2020-07-31T16:13:42.700+0300 7f0b2fca3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-228> 2020-07-31T16:13:42.700+0300 7f0b2fca3700 10 monclient(hunting): _init_auth method 2
-227> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient(hunting): handle_auth_done global_id 12838351 payload 450
-226> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient: _finish_hunting 0
-225> 2020-07-31T16:13:42.700+0300 7f0b304a4700 1 monclient: found mon.noname-f
-224> 2020-07-31T16:13:42.700+0300 7f0b304a4700 10 monclient: _send_mon_message to mon.noname-f at v2:192.168.5.15:3300/0
-223> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: handle_monmap mon_map magic: 0 v1
-222> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: got monmap 28 from mon.noname-f (according to old e28)
-221> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-220> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: _finish_auth 0
-219> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:13:12.708646+0300)
-218> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: _send_mon_message to mon.ripper5 at v2:192.168.5.15:3300/0
-217> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 5 monclient: authenticate success, global_id 12838351
-216> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: handle_config config(0 keys) v1
-215> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient: get_monmap_and_config success
-214> 2020-07-31T16:13:42.700+0300 7f0b314ffe00 10 monclient: shutdown
-213> 2020-07-31T16:13:42.700+0300 7f0b2dc9f700 4 set_mon_vals no callback set
-212> 2020-07-31T16:13:42.700+0300 7f0b2f4a2700 10 monclient: discarding stray monitor message mon_map magic: 0 v1
-211> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1169222
-209> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command 0 hook 0x5605b11d42d0
-205> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command version hook 0x5605b11d42d0
-204> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command git_version hook 0x5605b11d42d0
-203> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command help hook 0x5605b11d64b0
-202> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command get_command_descriptions hook 0x5605b11d6030
-201> 2020-07-31T16:13:42.704+0300 7f0b2eca1700 5 asok(0x5605b1278000) entry start
-200> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 1 bdev(0x5605b1e56380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:13:42.704+0300 7f0b314ffe00 1 bdev(0x5605b1e56380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore bluefs available hook 0x5605b11d6070
-196> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore bluefs stats hook 0x5605b11d6070
-195> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:13:42.708+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:13:43.020+0300 7f0b314ffe00 1 bdev(0x5605b1e56380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding auth protocol: cephx
-187> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding auth protocol: cephx
-186> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding auth protocol: cephx
-185> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: secure
-184> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: crc
-183> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: secure
-182> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: crc
-181> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: secure
-180> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: crc
-179> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: crc
-178> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: secure
-177> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: crc
-176> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: secure
-175> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: crc
-174> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1266940) adding con mode: secure
-173> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding auth protocol: cephx
-171> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding auth protocol: cephx
-170> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding auth protocol: cephx
-169> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: secure
-168> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: crc
-167> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: secure
-166> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: crc
-165> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: secure
-164> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: crc
-163> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: crc
-162> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: secure
-161> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: crc
-160> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: secure
-159> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: crc
-158> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1265940) adding con mode: secure
-157> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding auth protocol: cephx
-155> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding auth protocol: cephx
-154> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding auth protocol: cephx
-153> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-152> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-151> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-150> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-149> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-148> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-147> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-146> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-145> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-144> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-143> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: crc
-142> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1264940) adding con mode: secure
-141> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding auth protocol: cephx
-139> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding auth protocol: cephx
-138> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding auth protocol: cephx
-137> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: secure
-136> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: crc
-135> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: secure
-134> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: crc
-133> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: secure
-132> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: crc
-131> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: crc
-130> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: secure
-129> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: crc
-128> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: secure
-127> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: crc
-126> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1267940) adding con mode: secure
-125> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding auth protocol: cephx
-123> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding auth protocol: cephx
-122> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding auth protocol: cephx
-121> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: secure
-120> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: crc
-119> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: secure
-118> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: crc
-117> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: secure
-116> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: crc
-115> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: crc
-114> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: secure
-113> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: crc
-112> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: secure
-111> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: crc
-110> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f28940) adding con mode: secure
-109> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding auth protocol: cephx
-107> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding auth protocol: cephx
-106> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding auth protocol: cephx
-105> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: secure
-104> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: crc
-103> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: secure
-102> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: crc
-101> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: secure
-100> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: crc
-99> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: crc
-98> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: secure
-97> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: crc
-96> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: secure
-95> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: crc
-94> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f29940) adding con mode: secure
-93> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding auth protocol: cephx
-91> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding auth protocol: cephx
-90> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding auth protocol: cephx
-89> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: secure
-88> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: crc
-87> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: secure
-86> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: crc
-85> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: secure
-84> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: crc
-83> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: crc
-82> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: secure
-81> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: crc
-80> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: secure
-79> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: crc
-78> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 5 AuthRegistry(0x5605b1f2a940) adding con mode: secure
-77> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:13:43.264+0300 7f0b314ffe00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:13:43.268+0300 7f0b314ffe00 -1 Falling back to public interface
-74> 2020-07-31T16:13:43.268+0300 7f0b314ffe00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:13:43.268+0300 7f0b314ffe00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:13:43.275640+0300
created 2020-07-31T16:13:43.275640+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:13:43.284+0300 7f0b314ffe00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:13:43.284+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command objecter_requests hook 0x5605b11d6950
-66> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e57180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore bluefs available hook 0x5605b11d6970
-60> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore bluefs stats hook 0x5605b11d6970
-59> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:13:43.584+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:13:43.588+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator dump bluefs-db hook 0x5605b1f414d0
-55> 2020-07-31T16:13:43.588+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator score bluefs-db hook 0x5605b1f414d0
-54> 2020-07-31T16:13:43.588+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator fragmentation bluefs-db hook 0x5605b1f414d0
-53> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:13:43.640+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:13:45.476+0300 7f0b314ffe00 1 get compressor snappy = 0
-38> 2020-07-31T16:13:45.476+0300 7f0b314ffe00 1 load compressor snappy
-37> 2020-07-31T16:13:45.476+0300 7f0b314ffe00 1 add compressor snappy 0x5605b225ad50
-36> 2020-07-31T16:13:45.476+0300 7f0b314ffe00 1 get compressor snappy = 0x5605b225ad50
-35> 2020-07-31T16:13:45.476+0300 7f0b314ffe00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:13:45.476+0300 7f0b314ffe00 1 get compressor snappy = 0x5605b225ad50
-33> 2020-07-31T16:13:45.480+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator dump block hook 0x5605b221b2c0
-32> 2020-07-31T16:13:45.480+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator score block hook 0x5605b221b2c0
-31> 2020-07-31T16:13:45.480+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator fragmentation block hook 0x5605b221b2c0
-30> 2020-07-31T16:13:45.900+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x5605b225ae70 EEXIST
-29> 2020-07-31T16:14:09.952+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:14:09.952+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:14:09.952+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:14:09.952+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:14:09.952+0300 7f0b314ffe00 5 asok(0x5605b1278000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:14:09.952+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore bluefs available hook 0x5605b11d6970
-22> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore bluefs stats hook 0x5605b11d6970
-21> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 1 bdev(0x5605b1e56a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator dump bluefs-db hook 0x5605b1f41440
-17> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator score bluefs-db hook 0x5605b1f41440
-16> 2020-07-31T16:14:10.156+0300 7f0b314ffe00 5 asok(0x5605b1278000) register_command bluestore allocator fragmentation bluefs-db hook 0x5605b1f41440
-15> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:14:10.204+0300 7f0b314ffe00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:14:12.020+0300 7f0b314ffe00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f0b314ffe00 time 2020-07-31T16:14:12.018568+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x5605a66e29b7]
2: (()+0x9e1bbf) [0x5605a66e2bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x5605a6e37918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x5605a6cf3c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x5605a6cf6318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x5605a6d0b2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x5605a6d0c2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x5605a6d3b8e1]
9: (OSD::init()+0x61f) [0x5605a67c8daf]
10: (main()+0x3938) [0x5605a6730fb8]
11: (__libc_start_main()+0xea) [0x7f0b31a21cca]
12: (_start()+0x2e) [0x5605a674605e]

0> 2020-07-31T16:14:12.028+0300 7f0b314ffe00 -1 *** Caught signal (Aborted) **
in thread 7f0b314ffe00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f0b31f64140]
2: (gsignal()+0x141) [0x7f0b31a36cb1]
3: (abort()+0x123) [0x7f0b31a20537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x5605a66e2a12]
5: (()+0x9e1bbf) [0x5605a66e2bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x5605a6e37918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x5605a6cf3c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x5605a6cf6318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x5605a6d0b2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x5605a6d0c2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x5605a6d3b8e1]
12: (OSD::init()+0x61f) [0x5605a67c8daf]
13: (main()+0x3938) [0x5605a6730fb8]
14: (__libc_start_main()+0xea) [0x7f0b31a21cca]
15: (_start()+0x2e) [0x5605a674605e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f0b2dc9f700 / signal_handler
7f0b2eca1700 / admin_socket
7f0b2f4a2700 / service
7f0b2fca3700 / msgr-worker-2
7f0b304a4700 / msgr-worker-1
7f0b314ffe00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:14:22.368+0300 7f23d222ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:14:22.368+0300 7f23d222ae00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1169390
2020-07-31T16:14:22.368+0300 7f23d222ae00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev(0x564fbbf1c380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev(0x564fbbf1c380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:22.372+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:22.372+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:22.680+0300 7f23d222ae00 1 bdev(0x564fbbf1c380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:22.932+0300 7f23d222ae00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:14:22.932+0300 7f23d222ae00 -1 Falling back to public interface
2020-07-31T16:14:22.940+0300 7f23d222ae00 0 load: jerasure load: lrc load: isa
2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:23.240+0300 7f23d222ae00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:14:23.240+0300 7f23d222ae00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:47.880+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:48.064+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:48.064+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:48.064+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:48.192+0300 7f23d222ae00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f23d222ae00 time 2020-07-31T16:14:48.191479+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x564faf6779b7]
2: (()+0x9e1bbf) [0x564faf677bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x564fafdcc918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x564fafc88c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x564fafc8b318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x564fafca02b5]
7: (BlueStore::_deferred_replay()+0x313) [0x564fafca12b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x564fafcd08e1]
9: (OSD::init()+0x61f) [0x564faf75ddaf]
10: (main()+0x3938) [0x564faf6c5fb8]
11: (__libc_start_main()+0xea) [0x7f23d274ccca]
12: (_start()+0x2e) [0x564faf6db05e]

2020-07-31T16:14:48.200+0300 7f23d222ae00 -1 *** Caught signal (Aborted) **
in thread 7f23d222ae00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f23d2c8f140]
2: (gsignal()+0x141) [0x7f23d2761cb1]
3: (abort()+0x123) [0x7f23d274b537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x564faf677a12]
5: (()+0x9e1bbf) [0x564faf677bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x564fafdcc918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x564fafc88c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x564fafc8b318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x564fafca02b5]
10: (BlueStore::_deferred_replay()+0x313) [0x564fafca12b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x564fafcd08e1]
12: (OSD::init()+0x61f) [0x564faf75ddaf]
13: (main()+0x3938) [0x564faf6c5fb8]
14: (__libc_start_main()+0xea) [0x7f23d274ccca]
15: (_start()+0x2e) [0x564faf6db05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-305> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command assert hook 0x564fbb29c570
-304> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command abort hook 0x564fbb29c570
-303> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command leak_some_memory hook 0x564fbb29c570
-302> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perfcounters_dump hook 0x564fbb29c570
-301> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command 1 hook 0x564fbb29c570
-300> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perf dump hook 0x564fbb29c570
-299> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perfcounters_schema hook 0x564fbb29c570
-298> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perf histogram dump hook 0x564fbb29c570
-297> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command 2 hook 0x564fbb29c570
-296> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perf schema hook 0x564fbb29c570
-295> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perf histogram schema hook 0x564fbb29c570
-294> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command perf reset hook 0x564fbb29c570
-293> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config show hook 0x564fbb29c570
-292> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config help hook 0x564fbb29c570
-291> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config set hook 0x564fbb29c570
-290> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config unset hook 0x564fbb29c570
-289> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config get hook 0x564fbb29c570
-288> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config diff hook 0x564fbb29c570
-287> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command config diff get hook 0x564fbb29c570
-286> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command injectargs hook 0x564fbb29c570
-285> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command log flush hook 0x564fbb29c570
-284> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command log dump hook 0x564fbb29c570
-283> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command log reopen hook 0x564fbb29c570
-282> 2020-07-31T16:14:22.360+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command dump_mempools hook 0x564fbbf12068
-281> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient: get_monmap_and_config
-280> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient: build_initial_monmap
-279> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:14:22.367506+0300
created 2020-07-31T16:14:22.367506+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-278> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding auth protocol: cephx
-277> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding auth protocol: cephx
-276> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding auth protocol: cephx
-275> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-274> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-273> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-272> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-271> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-270> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-269> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-268> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-267> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-266> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-265> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-264> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-263> 2020-07-31T16:14:22.364+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-262> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient: init
-261> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding auth protocol: cephx
-260> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding auth protocol: cephx
-259> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding auth protocol: cephx
-258> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: secure
-257> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: crc
-256> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: secure
-255> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: crc
-254> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: secure
-253> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: crc
-252> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: crc
-251> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: secure
-250> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: crc
-249> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: secure
-248> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: crc
-247> 2020-07-31T16:14:22.364+0300 7f23d222ae00 5 AuthRegistry(0x7fff169dadb0) adding con mode: secure
-246> 2020-07-31T16:14:22.364+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-245> 2020-07-31T16:14:22.364+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-244> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient: _reopen_session rank -1
-243> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient: _add_conns ranks=[12,9,5,8,13,10,6,2,0,1,4,11,3,7]
-242> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): picked mon.noname-b con 0x564fbb33cc00 addr v2:192.168.5.21:3300/0
-241> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): picked mon.noname-f con 0x564fbb33d000 addr v2:192.168.5.15:3300/0
-240> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): picked mon.noname-b-legacy con 0x564fbb33d400 addr v1:192.168.5.21:6789/0
-239> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): start opening mon connection
-238> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): start opening mon connection
-237> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): _renew_subs
-236> 2020-07-31T16:14:22.364+0300 7f23d222ae00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:19:22.368600+0300
-235> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): get_auth_request con 0x564fbb33d000 auth_method 0
-234> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-233> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): _init_auth method 2
-232> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): handle_auth_reply_more payload 9
-231> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-230> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-229> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient(hunting): handle_auth_done global_id 12838358 payload 450
-228> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient: _finish_hunting 0
-227> 2020-07-31T16:14:22.368+0300 7f23d09ce700 1 monclient: found mon.noname-f
-226> 2020-07-31T16:14:22.368+0300 7f23d09ce700 10 monclient: _send_mon_message to mon.noname-f at v2:192.168.5.15:3300/0
-225> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: handle_monmap mon_map magic: 0 v1
-224> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: got monmap 28 from mon.noname-f (according to old e28)
-223> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-222> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: _finish_auth 0
-221> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:13:52.369565+0300)
-220> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: _send_mon_message to mon.ripper5 at v2:192.168.5.15:3300/0
-219> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: handle_config config(0 keys) v1
-218> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2020-07-31T16:14:22.368+0300 7f23ce9ca700 4 set_mon_vals no callback set
-216> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: got monmap 28 from mon.ripper5 (according to old e28)
-215> 2020-07-31T16:14:22.368+0300 7f23d01cd700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-214> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 monclient: authenticate success, global_id 12838358
-213> 2020-07-31T16:14:22.368+0300 7f23d222ae00 10 monclient: get_monmap_and_config success
-212> 2020-07-31T16:14:22.368+0300 7f23d222ae00 10 monclient: shutdown
-211> 2020-07-31T16:14:22.368+0300 7f23d222ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:14:22.368+0300 7f23d222ae00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1169390
-209> 2020-07-31T16:14:22.368+0300 7f23d222ae00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command 0 hook 0x564fbb29a2d0
-205> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command version hook 0x564fbb29a2d0
-204> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command git_version hook 0x564fbb29a2d0
-203> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command help hook 0x564fbb29c500
-202> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command get_command_descriptions hook 0x564fbb29c030
-201> 2020-07-31T16:14:22.368+0300 7f23cf9cc700 5 asok(0x564fbb33e000) entry start
-200> 2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev(0x564fbbf1c380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev(0x564fbbf1c380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore bluefs available hook 0x564fbb29c070
-196> 2020-07-31T16:14:22.368+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore bluefs stats hook 0x564fbb29c070
-195> 2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:14:22.368+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:14:22.372+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:14:22.372+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:14:22.372+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:14:22.372+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:14:22.680+0300 7f23d222ae00 1 bdev(0x564fbbf1c380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding auth protocol: cephx
-187> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding auth protocol: cephx
-186> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding auth protocol: cephx
-185> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: secure
-184> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: crc
-183> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: secure
-182> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: crc
-181> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: secure
-180> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: crc
-179> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: crc
-178> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: secure
-177> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: crc
-176> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: secure
-175> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: crc
-174> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32c940) adding con mode: secure
-173> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding auth protocol: cephx
-171> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding auth protocol: cephx
-170> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding auth protocol: cephx
-169> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: secure
-168> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: crc
-167> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: secure
-166> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: crc
-165> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: secure
-164> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: crc
-163> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: crc
-162> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: secure
-161> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: crc
-160> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: secure
-159> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: crc
-158> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32b940) adding con mode: secure
-157> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding auth protocol: cephx
-155> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding auth protocol: cephx
-154> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding auth protocol: cephx
-153> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-152> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-151> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-150> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-149> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-148> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-147> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-146> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-145> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-144> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-143> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: crc
-142> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32a940) adding con mode: secure
-141> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding auth protocol: cephx
-139> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding auth protocol: cephx
-138> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding auth protocol: cephx
-137> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: secure
-136> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: crc
-135> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: secure
-134> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: crc
-133> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: secure
-132> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: crc
-131> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: crc
-130> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: secure
-129> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: crc
-128> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: secure
-127> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: crc
-126> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbb32d940) adding con mode: secure
-125> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding auth protocol: cephx
-123> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding auth protocol: cephx
-122> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding auth protocol: cephx
-121> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: secure
-120> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: crc
-119> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: secure
-118> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: crc
-117> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: secure
-116> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: crc
-115> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: crc
-114> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: secure
-113> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: crc
-112> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: secure
-111> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: crc
-110> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe4940) adding con mode: secure
-109> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding auth protocol: cephx
-107> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding auth protocol: cephx
-106> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding auth protocol: cephx
-105> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: secure
-104> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: crc
-103> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: secure
-102> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: crc
-101> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: secure
-100> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: crc
-99> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: crc
-98> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: secure
-97> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: crc
-96> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: secure
-95> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: crc
-94> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe5940) adding con mode: secure
-93> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding auth protocol: cephx
-91> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding auth protocol: cephx
-90> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding auth protocol: cephx
-89> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: secure
-88> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: crc
-87> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: secure
-86> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: crc
-85> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: secure
-84> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: crc
-83> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: crc
-82> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: secure
-81> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: crc
-80> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: secure
-79> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: crc
-78> 2020-07-31T16:14:22.932+0300 7f23d222ae00 5 AuthRegistry(0x564fbbfe6940) adding con mode: secure
-77> 2020-07-31T16:14:22.932+0300 7f23d222ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:14:22.932+0300 7f23d222ae00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:14:22.932+0300 7f23d222ae00 -1 Falling back to public interface
-74> 2020-07-31T16:14:22.936+0300 7f23d222ae00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:14:22.936+0300 7f23d222ae00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:14:22.937350+0300
created 2020-07-31T16:14:22.937350+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:14:22.940+0300 7f23d222ae00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:14:22.940+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:14:23.240+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command objecter_requests hook 0x564fbb29c950
-66> 2020-07-31T16:14:23.240+0300 7f23d222ae00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:14:23.240+0300 7f23d222ae00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:14:23.240+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore bluefs available hook 0x564fbb29c970
-60> 2020-07-31T16:14:23.240+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore bluefs stats hook 0x564fbb29c970
-59> 2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:14:23.240+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:14:23.240+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator dump bluefs-db hook 0x564fbbffd4a0
-55> 2020-07-31T16:14:23.240+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator score bluefs-db hook 0x564fbbffd4a0
-54> 2020-07-31T16:14:23.240+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator fragmentation bluefs-db hook 0x564fbbffd4a0
-53> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:14:23.296+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:14:23.320+0300 7f23d222ae00 1 get compressor snappy = 0
-38> 2020-07-31T16:14:23.320+0300 7f23d222ae00 1 load compressor snappy
-37> 2020-07-31T16:14:23.320+0300 7f23d222ae00 1 add compressor snappy 0x564fbbffdcb0
-36> 2020-07-31T16:14:23.320+0300 7f23d222ae00 1 get compressor snappy = 0x564fbbffdcb0
-35> 2020-07-31T16:14:23.320+0300 7f23d222ae00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:14:23.320+0300 7f23d222ae00 1 get compressor snappy = 0x564fbbffdcb0
-33> 2020-07-31T16:14:23.324+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator dump block hook 0x564fbc4ed950
-32> 2020-07-31T16:14:23.324+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator score block hook 0x564fbc4ed950
-31> 2020-07-31T16:14:23.324+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator fragmentation block hook 0x564fbc4ed950
-30> 2020-07-31T16:14:23.784+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x564fbcfc7b00 EEXIST
-29> 2020-07-31T16:14:47.880+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:14:47.880+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:14:47.880+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:14:47.880+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:14:47.880+0300 7f23d222ae00 5 asok(0x564fbb33e000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:14:47.880+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:14:48.064+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore bluefs available hook 0x564fbb29c970
-22> 2020-07-31T16:14:48.064+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore bluefs stats hook 0x564fbb29c970
-21> 2020-07-31T16:14:48.064+0300 7f23d222ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:14:48.064+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:14:48.064+0300 7f23d222ae00 1 bdev(0x564fbbf1ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:14:48.064+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator dump bluefs-db hook 0x564fbbffd410
-17> 2020-07-31T16:14:48.064+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator score bluefs-db hook 0x564fbbffd410
-16> 2020-07-31T16:14:48.064+0300 7f23d222ae00 5 asok(0x564fbb33e000) register_command bluestore allocator fragmentation bluefs-db hook 0x564fbbffd410
-15> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:14:48.116+0300 7f23d222ae00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:14:48.192+0300 7f23d222ae00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f23d222ae00 time 2020-07-31T16:14:48.191479+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x564faf6779b7]
2: (()+0x9e1bbf) [0x564faf677bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x564fafdcc918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x564fafc88c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x564fafc8b318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x564fafca02b5]
7: (BlueStore::_deferred_replay()+0x313) [0x564fafca12b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x564fafcd08e1]
9: (OSD::init()+0x61f) [0x564faf75ddaf]
10: (main()+0x3938) [0x564faf6c5fb8]
11: (__libc_start_main()+0xea) [0x7f23d274ccca]
12: (_start()+0x2e) [0x564faf6db05e]

0> 2020-07-31T16:14:48.200+0300 7f23d222ae00 -1 *** Caught signal (Aborted) **
in thread 7f23d222ae00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f23d2c8f140]
2: (gsignal()+0x141) [0x7f23d2761cb1]
3: (abort()+0x123) [0x7f23d274b537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x564faf677a12]
5: (()+0x9e1bbf) [0x564faf677bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x564fafdcc918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x564fafc88c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x564fafc8b318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x564fafca02b5]
10: (BlueStore::_deferred_replay()+0x313) [0x564fafca12b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x564fafcd08e1]
12: (OSD::init()+0x61f) [0x564faf75ddaf]
13: (main()+0x3938) [0x564faf6c5fb8]
14: (__libc_start_main()+0xea) [0x7f23d274ccca]
15: (_start()+0x2e) [0x564faf6db05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f23ce9ca700 / signal_handler
7f23cf9cc700 / admin_socket
7f23d01cd700 / service
7f23d09ce700 / msgr-worker-2
7f23d222ae00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:14:58.424+0300 7f5a6415ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:14:58.424+0300 7f5a6415ae00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1169711
2020-07-31T16:14:58.424+0300 7f5a6415ae00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:58.740+0300 7f5a6415ae00 1 bdev(0x563824db8380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:58.984+0300 7f5a6415ae00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:14:58.984+0300 7f5a6415ae00 -1 Falling back to public interface
2020-07-31T16:14:58.992+0300 7f5a6415ae00 0 load: jerasure load: lrc load: isa
2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:14:59.300+0300 7f5a6415ae00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:14:59.300+0300 7f5a6415ae00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:15:25.596+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:15:25.872+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:15:25.872+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:15:25.872+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:15:26.000+0300 7f5a6415ae00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f5a6415ae00 time 2020-07-31T16:15:25.997428+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x56381875a9b7]
2: (()+0x9e1bbf) [0x56381875abbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x563818eaf918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x563818d6bc3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x563818d6e318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x563818d832b5]
7: (BlueStore::_deferred_replay()+0x313) [0x563818d842b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x563818db38e1]
9: (OSD::init()+0x61f) [0x563818840daf]
10: (main()+0x3938) [0x5638187a8fb8]
11: (__libc_start_main()+0xea) [0x7f5a6467ccca]
12: (_start()+0x2e) [0x5638187be05e]

2020-07-31T16:15:26.004+0300 7f5a6415ae00 -1 *** Caught signal (Aborted) **
in thread 7f5a6415ae00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f5a64bbf140]
2: (gsignal()+0x141) [0x7f5a64691cb1]
3: (abort()+0x123) [0x7f5a6467b537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x56381875aa12]
5: (()+0x9e1bbf) [0x56381875abbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x563818eaf918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x563818d6bc3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x563818d6e318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x563818d832b5]
10: (BlueStore::_deferred_replay()+0x313) [0x563818d842b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x563818db38e1]
12: (OSD::init()+0x61f) [0x563818840daf]
13: (main()+0x3938) [0x5638187a8fb8]
14: (__libc_start_main()+0xea) [0x7f5a6467ccca]
15: (_start()+0x2e) [0x5638187be05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-310> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command assert hook 0x563824138570
-309> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command abort hook 0x563824138570
-308> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command leak_some_memory hook 0x563824138570
-307> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perfcounters_dump hook 0x563824138570
-306> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command 1 hook 0x563824138570
-305> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perf dump hook 0x563824138570
-304> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perfcounters_schema hook 0x563824138570
-303> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perf histogram dump hook 0x563824138570
-302> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command 2 hook 0x563824138570
-301> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perf schema hook 0x563824138570
-300> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perf histogram schema hook 0x563824138570
-299> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command perf reset hook 0x563824138570
-298> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config show hook 0x563824138570
-297> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config help hook 0x563824138570
-296> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config set hook 0x563824138570
-295> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config unset hook 0x563824138570
-294> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config get hook 0x563824138570
-293> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config diff hook 0x563824138570
-292> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command config diff get hook 0x563824138570
-291> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command injectargs hook 0x563824138570
-290> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command log flush hook 0x563824138570
-289> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command log dump hook 0x563824138570
-288> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command log reopen hook 0x563824138570
-287> 2020-07-31T16:14:58.412+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command dump_mempools hook 0x563824dae068
-286> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: get_monmap_and_config
-285> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: build_initial_monmap
-284> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:14:58.422427+0300
created 2020-07-31T16:14:58.422427+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-283> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding auth protocol: cephx
-282> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding auth protocol: cephx
-281> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding auth protocol: cephx
-280> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-279> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-278> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-277> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-276> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-275> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-274> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-273> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-272> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-271> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-270> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-269> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-268> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-267> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: init
-266> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding auth protocol: cephx
-265> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding auth protocol: cephx
-264> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding auth protocol: cephx
-263> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: secure
-262> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: crc
-261> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: secure
-260> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: crc
-259> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: secure
-258> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: crc
-257> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: crc
-256> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: secure
-255> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: crc
-254> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: secure
-253> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: crc
-252> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 AuthRegistry(0x7ffcc2fec490) adding con mode: secure
-251> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-250> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-249> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: _reopen_session rank -1
-248> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: _add_conns ranks=[8,10,9,5,7,13,0,3,2,11,12,6,1,4]
-247> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): picked mon.noname-e con 0x5638241d8c00 addr v2:192.168.5.14:3300/0
-246> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): picked mon.noname-g con 0x5638241d9000 addr v2:192.168.5.16:3300/0
-245> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): picked mon.noname-f con 0x5638241d9400 addr v2:192.168.5.15:3300/0
-244> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): start opening mon connection
-243> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): start opening mon connection
-242> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): start opening mon connection
-241> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): _renew_subs
-240> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:19:58.423508+0300
-239> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): get_auth_request con 0x5638241d9400 auth_method 0
-238> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-237> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): _init_auth method 2
-236> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): handle_auth_reply_more payload 9
-235> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-234> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-233> 2020-07-31T16:14:58.420+0300 7f5a630ff700 10 monclient(hunting): get_auth_request con 0x5638241d8c00 auth_method 0
-232> 2020-07-31T16:14:58.420+0300 7f5a630ff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-231> 2020-07-31T16:14:58.420+0300 7f5a630ff700 10 monclient(hunting): _init_auth method 2
-230> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient(hunting): handle_auth_done global_id 12838365 payload 450
-229> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient: _finish_hunting 0
-228> 2020-07-31T16:14:58.420+0300 7f5a63900700 1 monclient: found mon.noname-f
-227> 2020-07-31T16:14:58.420+0300 7f5a63900700 10 monclient: _send_mon_message to mon.noname-f at v2:192.168.5.15:3300/0
-226> 2020-07-31T16:14:58.420+0300 7f5a628fe700 10 monclient: get_auth_request con 0x5638241d9000 auth_method 0
-225> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: handle_monmap mon_map magic: 0 v1
-224> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: got monmap 28 from mon.noname-f (according to old e28)
-223> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-222> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: _finish_auth 0
-221> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:14:28.424613+0300)
-220> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: _send_mon_message to mon.ripper5 at v2:192.168.5.15:3300/0
-219> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: handle_config config(0 keys) v1
-218> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: got monmap 28 from mon.ripper5 (according to old e28)
-216> 2020-07-31T16:14:58.420+0300 7f5a620fd700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-215> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 5 monclient: authenticate success, global_id 12838365
-214> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: get_monmap_and_config success
-213> 2020-07-31T16:14:58.420+0300 7f5a6415ae00 10 monclient: shutdown
-212> 2020-07-31T16:14:58.420+0300 7f5a608fa700 4 set_mon_vals no callback set
-211> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1169711
-209> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command 0 hook 0x5638241362d0
-205> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command version hook 0x5638241362d0
-204> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command git_version hook 0x5638241362d0
-203> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command help hook 0x5638241384b0
-202> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command get_command_descriptions hook 0x563824138030
-201> 2020-07-31T16:14:58.424+0300 7f5a618fc700 5 asok(0x5638241da000) entry start
-200> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore bluefs available hook 0x563824138070
-196> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore bluefs stats hook 0x563824138070
-195> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:14:58.424+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:14:58.740+0300 7f5a6415ae00 1 bdev(0x563824db8380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding auth protocol: cephx
-187> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding auth protocol: cephx
-186> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding auth protocol: cephx
-185> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-184> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-183> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-182> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-181> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-180> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-179> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-178> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-177> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-176> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-175> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: crc
-174> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c6940) adding con mode: secure
-173> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding auth protocol: cephx
-171> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding auth protocol: cephx
-170> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding auth protocol: cephx
-169> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: secure
-168> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: crc
-167> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: secure
-166> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: crc
-165> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: secure
-164> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: crc
-163> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: crc
-162> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: secure
-161> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: crc
-160> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: secure
-159> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: crc
-158> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c7940) adding con mode: secure
-157> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding auth protocol: cephx
-155> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding auth protocol: cephx
-154> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding auth protocol: cephx
-153> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: secure
-152> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: crc
-151> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: secure
-150> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: crc
-149> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: secure
-148> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: crc
-147> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: crc
-146> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: secure
-145> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: crc
-144> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: secure
-143> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: crc
-142> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c8940) adding con mode: secure
-141> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding auth protocol: cephx
-139> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding auth protocol: cephx
-138> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding auth protocol: cephx
-137> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: secure
-136> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: crc
-135> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: secure
-134> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: crc
-133> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: secure
-132> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: crc
-131> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: crc
-130> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: secure
-129> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: crc
-128> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: secure
-127> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: crc
-126> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x5638241c9940) adding con mode: secure
-125> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding auth protocol: cephx
-123> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding auth protocol: cephx
-122> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding auth protocol: cephx
-121> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: secure
-120> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: crc
-119> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: secure
-118> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: crc
-117> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: secure
-116> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: crc
-115> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: crc
-114> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: secure
-113> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: crc
-112> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: secure
-111> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: crc
-110> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e98940) adding con mode: secure
-109> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding auth protocol: cephx
-107> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding auth protocol: cephx
-106> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding auth protocol: cephx
-105> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: secure
-104> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: crc
-103> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: secure
-102> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: crc
-101> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: secure
-100> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: crc
-99> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: crc
-98> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: secure
-97> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: crc
-96> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: secure
-95> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: crc
-94> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e99940) adding con mode: secure
-93> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding auth protocol: cephx
-91> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding auth protocol: cephx
-90> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding auth protocol: cephx
-89> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: secure
-88> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: crc
-87> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: secure
-86> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: crc
-85> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: secure
-84> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: crc
-83> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: crc
-82> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: secure
-81> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: crc
-80> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: secure
-79> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: crc
-78> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 5 AuthRegistry(0x563824e9a940) adding con mode: secure
-77> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:14:58.984+0300 7f5a6415ae00 -1 Falling back to public interface
-74> 2020-07-31T16:14:58.988+0300 7f5a6415ae00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:14:58.988+0300 7f5a6415ae00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:14:58.989843+0300
created 2020-07-31T16:14:58.989843+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:14:58.992+0300 7f5a6415ae00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:14:58.992+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command objecter_requests hook 0x563824138940
-66> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db9180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore bluefs available hook 0x563824138960
-60> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore bluefs stats hook 0x563824138960
-59> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator dump bluefs-db hook 0x563824ed3500
-55> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator score bluefs-db hook 0x563824ed3500
-54> 2020-07-31T16:14:59.300+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator fragmentation bluefs-db hook 0x563824ed3500
-53> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:14:59.368+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 1 get compressor snappy = 0
-38> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 1 load compressor snappy
-37> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 1 add compressor snappy 0x563824ed3da0
-36> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 1 get compressor snappy = 0x563824ed3da0
-35> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 1 get compressor snappy = 0x563824ed3da0
-33> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator dump block hook 0x563825436ed0
-32> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator score block hook 0x563825436ed0
-31> 2020-07-31T16:14:59.392+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator fragmentation block hook 0x563825436ed0
-30> 2020-07-31T16:14:59.916+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x5638252f61b0 EEXIST
-29> 2020-07-31T16:15:25.596+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:15:25.596+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:15:25.596+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:15:25.596+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:15:25.596+0300 7f5a6415ae00 5 asok(0x5638241da000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:15:25.596+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore bluefs available hook 0x563824138960
-22> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore bluefs stats hook 0x563824138960
-21> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 1 bdev(0x563824db8a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator dump bluefs-db hook 0x563824ed3470
-17> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator score bluefs-db hook 0x563824ed3470
-16> 2020-07-31T16:15:25.872+0300 7f5a6415ae00 5 asok(0x5638241da000) register_command bluestore allocator fragmentation bluefs-db hook 0x563824ed3470
-15> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:15:25.924+0300 7f5a6415ae00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:15:26.000+0300 7f5a6415ae00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f5a6415ae00 time 2020-07-31T16:15:25.997428+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x56381875a9b7]
2: (()+0x9e1bbf) [0x56381875abbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x563818eaf918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x563818d6bc3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x563818d6e318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x563818d832b5]
7: (BlueStore::_deferred_replay()+0x313) [0x563818d842b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x563818db38e1]
9: (OSD::init()+0x61f) [0x563818840daf]
10: (main()+0x3938) [0x5638187a8fb8]
11: (__libc_start_main()+0xea) [0x7f5a6467ccca]
12: (_start()+0x2e) [0x5638187be05e]

0> 2020-07-31T16:15:26.004+0300 7f5a6415ae00 -1 *** Caught signal (Aborted) **
in thread 7f5a6415ae00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f5a64bbf140]
2: (gsignal()+0x141) [0x7f5a64691cb1]
3: (abort()+0x123) [0x7f5a6467b537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x56381875aa12]
5: (()+0x9e1bbf) [0x56381875abbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x563818eaf918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x563818d6bc3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x563818d6e318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x563818d832b5]
10: (BlueStore::_deferred_replay()+0x313) [0x563818d842b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x563818db38e1]
12: (OSD::init()+0x61f) [0x563818840daf]
13: (main()+0x3938) [0x5638187a8fb8]
14: (__libc_start_main()+0xea) [0x7f5a6467ccca]
15: (_start()+0x2e) [0x5638187be05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f5a608fa700 / signal_handler
7f5a618fc700 / admin_socket
7f5a620fd700 / service
7f5a628fe700 / msgr-worker-2
7f5a630ff700 / msgr-worker-1
7f5a63900700 / msgr-worker-0
7f5a6415ae00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1170636
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fe380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fe380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:08.608+0300 7f2eb08e6e00 1 bdev(0x559bdc2fe380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:08.872+0300 7f2eb08e6e00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:17:08.872+0300 7f2eb08e6e00 -1 Falling back to public interface
2020-07-31T16:17:08.880+0300 7f2eb08e6e00 0 load: jerasure load: lrc load: isa
2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:17:33.227+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:33.523+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:33.523+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:33.523+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:17:33.647+0300 7f2eb08e6e00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f2eb08e6e00 time 2020-07-31T16:17:33.647612+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x559bd15239b7]
2: (()+0x9e1bbf) [0x559bd1523bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x559bd1c78918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x559bd1b34c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x559bd1b37318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x559bd1b4c2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x559bd1b4d2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x559bd1b7c8e1]
9: (OSD::init()+0x61f) [0x559bd1609daf]
10: (main()+0x3938) [0x559bd1571fb8]
11: (__libc_start_main()+0xea) [0x7f2eb0e08cca]
12: (_start()+0x2e) [0x559bd158705e]

2020-07-31T16:17:33.655+0300 7f2eb08e6e00 -1 *** Caught signal (Aborted) **
in thread 7f2eb08e6e00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f2eb134b140]
2: (gsignal()+0x141) [0x7f2eb0e1dcb1]
3: (abort()+0x123) [0x7f2eb0e07537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x559bd1523a12]
5: (()+0x9e1bbf) [0x559bd1523bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x559bd1c78918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x559bd1b34c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x559bd1b37318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x559bd1b4c2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x559bd1b4d2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x559bd1b7c8e1]
12: (OSD::init()+0x61f) [0x559bd1609daf]
13: (main()+0x3938) [0x559bd1571fb8]
14: (__libc_start_main()+0xea) [0x7f2eb0e08cca]
15: (_start()+0x2e) [0x559bd158705e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-318> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command assert hook 0x559bdb67e570
-317> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command abort hook 0x559bdb67e570
-316> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command leak_some_memory hook 0x559bdb67e570
-315> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perfcounters_dump hook 0x559bdb67e570
-314> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command 1 hook 0x559bdb67e570
-313> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perf dump hook 0x559bdb67e570
-312> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perfcounters_schema hook 0x559bdb67e570
-311> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perf histogram dump hook 0x559bdb67e570
-310> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command 2 hook 0x559bdb67e570
-309> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perf schema hook 0x559bdb67e570
-308> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perf histogram schema hook 0x559bdb67e570
-307> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command perf reset hook 0x559bdb67e570
-306> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config show hook 0x559bdb67e570
-305> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config help hook 0x559bdb67e570
-304> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config set hook 0x559bdb67e570
-303> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config unset hook 0x559bdb67e570
-302> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config get hook 0x559bdb67e570
-301> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config diff hook 0x559bdb67e570
-300> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command config diff get hook 0x559bdb67e570
-299> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command injectargs hook 0x559bdb67e570
-298> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command log flush hook 0x559bdb67e570
-297> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command log dump hook 0x559bdb67e570
-296> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command log reopen hook 0x559bdb67e570
-295> 2020-07-31T16:17:08.300+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command dump_mempools hook 0x559bdc2f4068
-294> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: get_monmap_and_config
-293> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: build_initial_monmap
-292> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:17:08.306515+0300
created 2020-07-31T16:17:08.306515+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-291> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding auth protocol: cephx
-290> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding auth protocol: cephx
-289> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding auth protocol: cephx
-288> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-287> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-286> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-285> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-284> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-283> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-282> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-281> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-280> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-279> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-278> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-277> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-276> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-275> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: init
-274> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding auth protocol: cephx
-273> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding auth protocol: cephx
-272> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding auth protocol: cephx
-271> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: secure
-270> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: crc
-269> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: secure
-268> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: crc
-267> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: secure
-266> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: crc
-265> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: crc
-264> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: secure
-263> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: crc
-262> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: secure
-261> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: crc
-260> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 AuthRegistry(0x7ffce48ef5f0) adding con mode: secure
-259> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-258> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-257> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: _reopen_session rank -1
-256> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: _add_conns ranks=[11,13,9,2,4,5,3,8,6,1,0,10,12,7]
-255> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): picked mon.noname-a con 0x559bdb71ec00 addr v2:192.168.5.20:3300/0
-254> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): picked mon.noname-c con 0x559bdb71f000 addr v2:192.168.5.22:3300/0
-253> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): picked mon.noname-f con 0x559bdb71f400 addr v2:192.168.5.15:3300/0
-252> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): start opening mon connection
-251> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): start opening mon connection
-250> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): start opening mon connection
-249> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): _renew_subs
-248> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:22:08.307539+0300
-247> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): get_auth_request con 0x559bdb71f400 auth_method 0
-246> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-245> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): _init_auth method 2
-244> 2020-07-31T16:17:08.304+0300 7f2eaf08a700 10 monclient(hunting): get_auth_request con 0x559bdb71f000 auth_method 0
-243> 2020-07-31T16:17:08.304+0300 7f2eaf08a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-242> 2020-07-31T16:17:08.304+0300 7f2eaf08a700 10 monclient(hunting): _init_auth method 2
-241> 2020-07-31T16:17:08.304+0300 7f2eaf88b700 10 monclient(hunting): get_auth_request con 0x559bdb71ec00 auth_method 0
-240> 2020-07-31T16:17:08.304+0300 7f2eaf88b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-239> 2020-07-31T16:17:08.304+0300 7f2eaf88b700 10 monclient(hunting): _init_auth method 2
-238> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): handle_auth_reply_more payload 9
-237> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-236> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-235> 2020-07-31T16:17:08.304+0300 7f2eaf08a700 10 monclient(hunting): handle_auth_reply_more payload 9
-234> 2020-07-31T16:17:08.304+0300 7f2eaf08a700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-233> 2020-07-31T16:17:08.304+0300 7f2eaf08a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-232> 2020-07-31T16:17:08.304+0300 7f2eaf88b700 10 monclient(hunting): handle_auth_reply_more payload 9
-231> 2020-07-31T16:17:08.304+0300 7f2eaf88b700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-230> 2020-07-31T16:17:08.304+0300 7f2eaf88b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-229> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient(hunting): handle_auth_done global_id 12838470 payload 450
-228> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient: _finish_hunting 0
-227> 2020-07-31T16:17:08.304+0300 7f2eb008c700 1 monclient: found mon.noname-f
-226> 2020-07-31T16:17:08.304+0300 7f2eb008c700 10 monclient: _send_mon_message to mon.noname-f at v2:192.168.5.15:3300/0
-225> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: handle_monmap mon_map magic: 0 v1
-224> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: got monmap 28 from mon.noname-f (according to old e28)
-223> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-222> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: _finish_auth 0
-221> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:16:38.308504+0300)
-220> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: _send_mon_message to mon.ripper5 at v2:192.168.5.15:3300/0
-219> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: handle_config config(0 keys) v1
-218> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2020-07-31T16:17:08.304+0300 7f2ead086700 4 set_mon_vals no callback set
-216> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: got monmap 28 from mon.ripper5 (according to old e28)
-215> 2020-07-31T16:17:08.304+0300 7f2eae889700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-214> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 5 monclient: authenticate success, global_id 12838470
-213> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: get_monmap_and_config success
-212> 2020-07-31T16:17:08.304+0300 7f2eb08e6e00 10 monclient: shutdown
-211> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1170636
-209> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command 0 hook 0x559bdb67c2d0
-205> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command version hook 0x559bdb67c2d0
-204> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command git_version hook 0x559bdb67c2d0
-203> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command help hook 0x559bdb67e370
-202> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command get_command_descriptions hook 0x559bdb67e030
-201> 2020-07-31T16:17:08.308+0300 7f2eae088700 5 asok(0x559bdb720000) entry start
-200> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fe380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fe380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore bluefs available hook 0x559bdb67e070
-196> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore bluefs stats hook 0x559bdb67e070
-195> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:17:08.308+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:17:08.608+0300 7f2eb08e6e00 1 bdev(0x559bdc2fe380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding auth protocol: cephx
-187> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding auth protocol: cephx
-186> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding auth protocol: cephx
-185> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-184> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-183> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-182> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-181> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-180> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-179> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-178> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-177> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-176> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-175> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: crc
-174> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70c940) adding con mode: secure
-173> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding auth protocol: cephx
-171> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding auth protocol: cephx
-170> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding auth protocol: cephx
-169> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: secure
-168> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: crc
-167> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: secure
-166> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: crc
-165> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: secure
-164> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: crc
-163> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: crc
-162> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: secure
-161> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: crc
-160> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: secure
-159> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: crc
-158> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70d940) adding con mode: secure
-157> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding auth protocol: cephx
-155> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding auth protocol: cephx
-154> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding auth protocol: cephx
-153> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: secure
-152> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: crc
-151> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: secure
-150> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: crc
-149> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: secure
-148> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: crc
-147> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: crc
-146> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: secure
-145> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: crc
-144> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: secure
-143> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: crc
-142> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70e940) adding con mode: secure
-141> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding auth protocol: cephx
-139> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding auth protocol: cephx
-138> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding auth protocol: cephx
-137> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: secure
-136> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: crc
-135> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: secure
-134> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: crc
-133> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: secure
-132> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: crc
-131> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: crc
-130> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: secure
-129> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: crc
-128> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: secure
-127> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: crc
-126> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdb70f940) adding con mode: secure
-125> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding auth protocol: cephx
-123> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding auth protocol: cephx
-122> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding auth protocol: cephx
-121> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: secure
-120> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: crc
-119> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: secure
-118> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: crc
-117> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: secure
-116> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: crc
-115> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: crc
-114> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: secure
-113> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: crc
-112> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: secure
-111> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: crc
-110> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e0940) adding con mode: secure
-109> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding auth protocol: cephx
-107> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding auth protocol: cephx
-106> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding auth protocol: cephx
-105> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: secure
-104> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: crc
-103> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: secure
-102> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: crc
-101> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: secure
-100> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: crc
-99> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: crc
-98> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: secure
-97> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: crc
-96> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: secure
-95> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: crc
-94> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e1940) adding con mode: secure
-93> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding auth protocol: cephx
-91> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding auth protocol: cephx
-90> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding auth protocol: cephx
-89> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: secure
-88> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: crc
-87> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: secure
-86> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: crc
-85> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: secure
-84> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: crc
-83> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: crc
-82> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: secure
-81> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: crc
-80> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: secure
-79> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: crc
-78> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 5 AuthRegistry(0x559bdc3e2940) adding con mode: secure
-77> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:17:08.872+0300 7f2eb08e6e00 -1 Falling back to public interface
-74> 2020-07-31T16:17:08.876+0300 7f2eb08e6e00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:17:08.876+0300 7f2eb08e6e00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:17:08.877668+0300
created 2020-07-31T16:17:08.877668+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:17:08.880+0300 7f2eb08e6e00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:17:08.880+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command objecter_requests hook 0x559bdb67e930
-66> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2ff180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore bluefs available hook 0x559bdb67e950
-60> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore bluefs stats hook 0x559bdb67e950
-59> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator dump bluefs-db hook 0x559bdc3f5500
-55> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator score bluefs-db hook 0x559bdc3f5500
-54> 2020-07-31T16:17:09.208+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator fragmentation bluefs-db hook 0x559bdc3f5500
-53> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:17:09.264+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:17:09.284+0300 7f2eb08e6e00 1 get compressor snappy = 0
-38> 2020-07-31T16:17:09.284+0300 7f2eb08e6e00 1 load compressor snappy
-37> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 1 add compressor snappy 0x559bdc3f57d0
-36> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 1 get compressor snappy = 0x559bdc3f57d0
-35> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 1 get compressor snappy = 0x559bdc3f57d0
-33> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator dump block hook 0x559bdc99e540
-32> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator score block hook 0x559bdc99e540
-31> 2020-07-31T16:17:09.288+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator fragmentation block hook 0x559bdc99e540
-30> 2020-07-31T16:17:09.720+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x559bdc8bf710 EEXIST
-29> 2020-07-31T16:17:33.227+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:17:33.227+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:17:33.227+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:17:33.227+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:17:33.227+0300 7f2eb08e6e00 5 asok(0x559bdb720000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:17:33.227+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore bluefs available hook 0x559bdb67e950
-22> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore bluefs stats hook 0x559bdb67e950
-21> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 1 bdev(0x559bdc2fea80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator dump bluefs-db hook 0x559bdc3f5470
-17> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator score bluefs-db hook 0x559bdc3f5470
-16> 2020-07-31T16:17:33.523+0300 7f2eb08e6e00 5 asok(0x559bdb720000) register_command bluestore allocator fragmentation bluefs-db hook 0x559bdc3f5470
-15> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:17:33.575+0300 7f2eb08e6e00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:17:33.647+0300 7f2eb08e6e00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f2eb08e6e00 time 2020-07-31T16:17:33.647612+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x559bd15239b7]
2: (()+0x9e1bbf) [0x559bd1523bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x559bd1c78918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x559bd1b34c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x559bd1b37318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x559bd1b4c2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x559bd1b4d2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x559bd1b7c8e1]
9: (OSD::init()+0x61f) [0x559bd1609daf]
10: (main()+0x3938) [0x559bd1571fb8]
11: (__libc_start_main()+0xea) [0x7f2eb0e08cca]
12: (_start()+0x2e) [0x559bd158705e]

0> 2020-07-31T16:17:33.655+0300 7f2eb08e6e00 -1 *** Caught signal (Aborted) **
in thread 7f2eb08e6e00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f2eb134b140]
2: (gsignal()+0x141) [0x7f2eb0e1dcb1]
3: (abort()+0x123) [0x7f2eb0e07537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x559bd1523a12]
5: (()+0x9e1bbf) [0x559bd1523bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x559bd1c78918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x559bd1b34c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x559bd1b37318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x559bd1b4c2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x559bd1b4d2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x559bd1b7c8e1]
12: (OSD::init()+0x61f) [0x559bd1609daf]
13: (main()+0x3938) [0x559bd1571fb8]
14: (__libc_start_main()+0xea) [0x7f2eb0e08cca]
15: (_start()+0x2e) [0x559bd158705e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f2ead086700 / signal_handler
7f2eae088700 / admin_socket
7f2eae889700 / service
7f2eaf08a700 / msgr-worker-2
7f2eaf88b700 / msgr-worker-1
7f2eb008c700 / msgr-worker-0
7f2eb08e6e00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:17:43.911+0300 7f8f715e1e00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:17:43.911+0300 7f8f715e1e00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1170968
2020-07-31T16:17:43.911+0300 7f8f715e1e00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:44.235+0300 7f8f715e1e00 1 bdev(0x55d2bc366380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:44.471+0300 7f8f715e1e00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:17:44.471+0300 7f8f715e1e00 -1 Falling back to public interface
2020-07-31T16:17:44.479+0300 7f8f715e1e00 0 load: jerasure load: lrc load: isa
2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:17:44.783+0300 7f8f715e1e00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:17:44.783+0300 7f8f715e1e00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:09.203+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:18:09.355+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:09.355+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:09.355+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:09.479+0300 7f8f715e1e00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f8f715e1e00 time 2020-07-31T16:18:09.479056+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x55d2b05009b7]
2: (()+0x9e1bbf) [0x55d2b0500bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55d2b0c55918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55d2b0b11c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x55d2b0b14318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x55d2b0b292b5]
7: (BlueStore::_deferred_replay()+0x313) [0x55d2b0b2a2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x55d2b0b598e1]
9: (OSD::init()+0x61f) [0x55d2b05e6daf]
10: (main()+0x3938) [0x55d2b054efb8]
11: (__libc_start_main()+0xea) [0x7f8f71b03cca]
12: (_start()+0x2e) [0x55d2b056405e]

2020-07-31T16:18:09.483+0300 7f8f715e1e00 -1 *** Caught signal (Aborted) **
in thread 7f8f715e1e00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f8f72046140]
2: (gsignal()+0x141) [0x7f8f71b18cb1]
3: (abort()+0x123) [0x7f8f71b02537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x55d2b0500a12]
5: (()+0x9e1bbf) [0x55d2b0500bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55d2b0c55918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55d2b0b11c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x55d2b0b14318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x55d2b0b292b5]
10: (BlueStore::_deferred_replay()+0x313) [0x55d2b0b2a2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x55d2b0b598e1]
12: (OSD::init()+0x61f) [0x55d2b05e6daf]
13: (main()+0x3938) [0x55d2b054efb8]
14: (__libc_start_main()+0xea) [0x7f8f71b03cca]
15: (_start()+0x2e) [0x55d2b056405e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-336> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command assert hook 0x55d2bb6e6570
-335> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command abort hook 0x55d2bb6e6570
-334> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command leak_some_memory hook 0x55d2bb6e6570
-333> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perfcounters_dump hook 0x55d2bb6e6570
-332> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command 1 hook 0x55d2bb6e6570
-331> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perf dump hook 0x55d2bb6e6570
-330> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perfcounters_schema hook 0x55d2bb6e6570
-329> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perf histogram dump hook 0x55d2bb6e6570
-328> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command 2 hook 0x55d2bb6e6570
-327> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perf schema hook 0x55d2bb6e6570
-326> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perf histogram schema hook 0x55d2bb6e6570
-325> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command perf reset hook 0x55d2bb6e6570
-324> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config show hook 0x55d2bb6e6570
-323> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config help hook 0x55d2bb6e6570
-322> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config set hook 0x55d2bb6e6570
-321> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config unset hook 0x55d2bb6e6570
-320> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config get hook 0x55d2bb6e6570
-319> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config diff hook 0x55d2bb6e6570
-318> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command config diff get hook 0x55d2bb6e6570
-317> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command injectargs hook 0x55d2bb6e6570
-316> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command log flush hook 0x55d2bb6e6570
-315> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command log dump hook 0x55d2bb6e6570
-314> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command log reopen hook 0x55d2bb6e6570
-313> 2020-07-31T16:17:43.903+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command dump_mempools hook 0x55d2bc35c068
-312> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient: get_monmap_and_config
-311> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient: build_initial_monmap
-310> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:17:43.911102+0300
created 2020-07-31T16:17:43.911102+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-309> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding auth protocol: cephx
-308> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding auth protocol: cephx
-307> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding auth protocol: cephx
-306> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-305> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-304> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-303> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-302> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-301> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-300> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-299> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-298> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-297> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-296> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-295> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-294> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-293> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient: init
-292> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding auth protocol: cephx
-291> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding auth protocol: cephx
-290> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding auth protocol: cephx
-289> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: secure
-288> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: crc
-287> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: secure
-286> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: crc
-285> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: secure
-284> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: crc
-283> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: crc
-282> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: secure
-281> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: crc
-280> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: secure
-279> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: crc
-278> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 5 AuthRegistry(0x7ffd4606d1c0) adding con mode: secure
-277> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-276> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-275> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient: _reopen_session rank -1
-274> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient: _add_conns ranks=[4,0,2,10,7,11,12,6,1,3,13,8,9,5]
-273> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient(hunting): picked mon.noname-a-legacy con 0x55d2bb786c00 addr v1:192.168.5.20:6789/0
-272> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient(hunting): picked mon.noname-d-legacy con 0x55d2bb787000 addr v1:192.168.5.11:6789/0
-271> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient(hunting): picked mon.noname-f-legacy con 0x55d2bb787400 addr v1:192.168.5.15:6789/0
-270> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient(hunting): _renew_subs
-269> 2020-07-31T16:17:43.907+0300 7f8f715e1e00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:22:43.911960+0300
-268> 2020-07-31T16:17:43.907+0300 7f8f6f584700 10 monclient(hunting): _init_auth method 2
-267> 2020-07-31T16:17:43.907+0300 7f8f6f584700 10 monclient(hunting): my global_id is 12838547
-266> 2020-07-31T16:17:43.907+0300 7f8f6f584700 10 monclient(hunting): _init_auth method 2
-265> 2020-07-31T16:17:43.907+0300 7f8f6f584700 10 monclient(hunting): my global_id is 12838538
-264> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _finish_hunting 0
-263> 2020-07-31T16:17:43.911+0300 7f8f6f584700 1 monclient: found mon.noname-f-legacy
-262> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:192.168.5.15:6789/0
-261> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _finish_auth 0
-260> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:17:13.912826+0300)
-259> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:192.168.5.15:6789/0
-258> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: handle_monmap mon_map magic: 0 v1
-257> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: got monmap 28 from mon.noname-f-legacy (according to old e28)
-256> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-255> 2020-07-31T16:17:43.911+0300 7f8f6f584700 1 monclient: mon.ripper5 has (v2) addrs [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] but i'm connected to v1:192.168.5.15:6789/0, reconnecting
-254> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _reopen_session rank -1
-253> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: _add_conns ranks=[5,4,2,1,6,0,3]
-252> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): picked mon.ivy3 con 0x55d2bb787c00 addr [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0]
-251> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): picked mon.ivy2 con 0x55d2bc406000 addr [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0]
-250> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): picked mon.ripper6 con 0x55d2bc406400 addr [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0]
-249> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): start opening mon connection
-248> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): start opening mon connection
-247> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): start opening mon connection
-246> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient(hunting): _renew_subs
-245> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): get_auth_request con 0x55d2bb787c00 auth_method 0
-244> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-243> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): _init_auth method 2
-242> 2020-07-31T16:17:43.911+0300 7f8f6fd85700 10 monclient(hunting): get_auth_request con 0x55d2bc406000 auth_method 0
-241> 2020-07-31T16:17:43.911+0300 7f8f6fd85700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-240> 2020-07-31T16:17:43.911+0300 7f8f6fd85700 10 monclient(hunting): _init_auth method 2
-239> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): handle_auth_reply_more payload 9
-238> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-237> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-236> 2020-07-31T16:17:43.911+0300 7f8f70d87700 10 monclient(hunting): get_auth_request con 0x55d2bc406400 auth_method 0
-235> 2020-07-31T16:17:43.911+0300 7f8f70d87700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-234> 2020-07-31T16:17:43.911+0300 7f8f70d87700 10 monclient(hunting): _init_auth method 2
-233> 2020-07-31T16:17:43.911+0300 7f8f6fd85700 10 monclient(hunting): handle_auth_reply_more payload 9
-232> 2020-07-31T16:17:43.911+0300 7f8f6fd85700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-231> 2020-07-31T16:17:43.911+0300 7f8f6fd85700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-230> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient(hunting): handle_auth_done global_id 12838547 payload 450
-229> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient: _finish_hunting 0
-228> 2020-07-31T16:17:43.911+0300 7f8f70586700 1 monclient: found mon.ivy3
-227> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient: _send_mon_message to mon.ivy3 at v2:192.168.5.22:3300/0
-226> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient: _finish_auth 0
-225> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:17:13.913658+0300)
-224> 2020-07-31T16:17:43.911+0300 7f8f70586700 10 monclient: _send_mon_message to mon.ivy3 at v2:192.168.5.22:3300/0
-223> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 monclient: authenticate success, global_id 12838547
-222> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: handle_monmap mon_map magic: 0 v1
-221> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: got monmap 28 from mon.ivy3 (according to old e28)
-220> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-219> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: handle_config config(0 keys) v1
-218> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: got monmap 28 from mon.ivy3 (according to old e28)
-216> 2020-07-31T16:17:43.911+0300 7f8f6dd81700 4 set_mon_vals no callback set
-215> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-214> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 10 monclient: get_monmap_and_config success
-213> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 10 monclient: shutdown
-212> 2020-07-31T16:17:43.911+0300 7f8f6f584700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1
-211> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1170968
-209> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command 0 hook 0x55d2bb6e42d0
-205> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command version hook 0x55d2bb6e42d0
-204> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command git_version hook 0x55d2bb6e42d0
-203> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command help hook 0x55d2bb6e68c0
-202> 2020-07-31T16:17:43.911+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command get_command_descriptions hook 0x55d2bb6e6030
-201> 2020-07-31T16:17:43.911+0300 7f8f6ed83700 5 asok(0x55d2bb788000) entry start
-200> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore bluefs available hook 0x55d2bb6e6070
-196> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore bluefs stats hook 0x55d2bb6e6070
-195> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:17:43.915+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:17:44.235+0300 7f8f715e1e00 1 bdev(0x55d2bc366380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding auth protocol: cephx
-187> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding auth protocol: cephx
-186> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding auth protocol: cephx
-185> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-184> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-183> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-182> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-181> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-180> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-179> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-178> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-177> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-176> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-175> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: crc
-174> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb774940) adding con mode: secure
-173> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding auth protocol: cephx
-171> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding auth protocol: cephx
-170> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding auth protocol: cephx
-169> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: secure
-168> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: crc
-167> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: secure
-166> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: crc
-165> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: secure
-164> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: crc
-163> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: crc
-162> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: secure
-161> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: crc
-160> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: secure
-159> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: crc
-158> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb776940) adding con mode: secure
-157> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding auth protocol: cephx
-155> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding auth protocol: cephx
-154> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding auth protocol: cephx
-153> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: secure
-152> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: crc
-151> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: secure
-150> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: crc
-149> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: secure
-148> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: crc
-147> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: crc
-146> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: secure
-145> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: crc
-144> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: secure
-143> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: crc
-142> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bb777140) adding con mode: secure
-141> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding auth protocol: cephx
-139> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding auth protocol: cephx
-138> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding auth protocol: cephx
-137> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: secure
-136> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: crc
-135> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: secure
-134> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: crc
-133> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: secure
-132> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: crc
-131> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: crc
-130> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: secure
-129> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: crc
-128> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: secure
-127> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: crc
-126> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43c140) adding con mode: secure
-125> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding auth protocol: cephx
-123> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding auth protocol: cephx
-122> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding auth protocol: cephx
-121> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: secure
-120> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: crc
-119> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: secure
-118> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: crc
-117> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: secure
-116> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: crc
-115> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: crc
-114> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: secure
-113> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: crc
-112> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: secure
-111> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: crc
-110> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43d140) adding con mode: secure
-109> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding auth protocol: cephx
-107> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding auth protocol: cephx
-106> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding auth protocol: cephx
-105> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: secure
-104> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: crc
-103> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: secure
-102> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: crc
-101> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: secure
-100> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: crc
-99> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: crc
-98> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: secure
-97> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: crc
-96> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: secure
-95> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: crc
-94> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43e140) adding con mode: secure
-93> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding auth protocol: cephx
-91> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding auth protocol: cephx
-90> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding auth protocol: cephx
-89> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: secure
-88> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: crc
-87> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: secure
-86> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: crc
-85> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: secure
-84> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: crc
-83> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: crc
-82> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: secure
-81> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: crc
-80> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: secure
-79> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: crc
-78> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 5 AuthRegistry(0x55d2bc43f140) adding con mode: secure
-77> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:17:44.471+0300 7f8f715e1e00 -1 Falling back to public interface
-74> 2020-07-31T16:17:44.475+0300 7f8f715e1e00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:17:44.475+0300 7f8f715e1e00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:17:44.477376+0300
created 2020-07-31T16:17:44.477376+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:17:44.479+0300 7f8f715e1e00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:17:44.479+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command objecter_requests hook 0x55d2bb6e6980
-66> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc367180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore bluefs available hook 0x55d2bb6e69a0
-60> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore bluefs stats hook 0x55d2bb6e69a0
-59> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator dump bluefs-db hook 0x55d2bc459410
-55> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator score bluefs-db hook 0x55d2bc459410
-54> 2020-07-31T16:17:44.783+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator fragmentation bluefs-db hook 0x55d2bc459410
-53> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:17:44.843+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 1 get compressor snappy = 0
-38> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 1 load compressor snappy
-37> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 1 add compressor snappy 0x55d2bc459860
-36> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 1 get compressor snappy = 0x55d2bc459860
-35> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 1 get compressor snappy = 0x55d2bc459860
-33> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator dump block hook 0x55d2bca04ff0
-32> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator score block hook 0x55d2bca04ff0
-31> 2020-07-31T16:17:44.867+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator fragmentation block hook 0x55d2bca04ff0
-30> 2020-07-31T16:17:45.295+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x55d2bc34c4e0 EEXIST
-29> 2020-07-31T16:18:09.199+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:18:09.199+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:18:09.199+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:18:09.203+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:18:09.203+0300 7f8f715e1e00 5 asok(0x55d2bb788000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:18:09.203+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore bluefs available hook 0x55d2bb6e69a0
-22> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore bluefs stats hook 0x55d2bb6e69a0
-21> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 1 bdev(0x55d2bc366a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator dump bluefs-db hook 0x55d2bc459380
-17> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator score bluefs-db hook 0x55d2bc459380
-16> 2020-07-31T16:18:09.355+0300 7f8f715e1e00 5 asok(0x55d2bb788000) register_command bluestore allocator fragmentation bluefs-db hook 0x55d2bc459380
-15> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:18:09.407+0300 7f8f715e1e00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:18:09.479+0300 7f8f715e1e00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f8f715e1e00 time 2020-07-31T16:18:09.479056+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x55d2b05009b7]
2: (()+0x9e1bbf) [0x55d2b0500bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55d2b0c55918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55d2b0b11c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x55d2b0b14318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x55d2b0b292b5]
7: (BlueStore::_deferred_replay()+0x313) [0x55d2b0b2a2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x55d2b0b598e1]
9: (OSD::init()+0x61f) [0x55d2b05e6daf]
10: (main()+0x3938) [0x55d2b054efb8]
11: (__libc_start_main()+0xea) [0x7f8f71b03cca]
12: (_start()+0x2e) [0x55d2b056405e]

0> 2020-07-31T16:18:09.483+0300 7f8f715e1e00 -1 *** Caught signal (Aborted) **
in thread 7f8f715e1e00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f8f72046140]
2: (gsignal()+0x141) [0x7f8f71b18cb1]
3: (abort()+0x123) [0x7f8f71b02537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x55d2b0500a12]
5: (()+0x9e1bbf) [0x55d2b0500bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55d2b0c55918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55d2b0b11c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x55d2b0b14318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x55d2b0b292b5]
10: (BlueStore::_deferred_replay()+0x313) [0x55d2b0b2a2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x55d2b0b598e1]
12: (OSD::init()+0x61f) [0x55d2b05e6daf]
13: (main()+0x3938) [0x55d2b054efb8]
14: (__libc_start_main()+0xea) [0x7f8f71b03cca]
15: (_start()+0x2e) [0x55d2b056405e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f8f6dd81700 / signal_handler
7f8f6ed83700 / admin_socket
7f8f6f584700 / service
7f8f6fd85700 / msgr-worker-2
7f8f70586700 / msgr-worker-1
7f8f70d87700 / msgr-worker-0
7f8f715e1e00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1171275
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev(0x560cbac36380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev(0x560cbac36380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:19.667+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:19.667+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:18:19.987+0300 7f0eaadc9e00 1 bdev(0x560cbac36380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:18:20.215+0300 7f0eaadc9e00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:18:20.215+0300 7f0eaadc9e00 -1 Falling back to public interface
2020-07-31T16:18:20.223+0300 7f0eaadc9e00 0 load: jerasure load: lrc load: isa
2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:45.043+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:18:45.123+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:18:45.123+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:18:45.123+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:18:45.247+0300 7f0eaadc9e00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f0eaadc9e00 time 2020-07-31T16:18:45.247918+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x560cae9339b7]
2: (()+0x9e1bbf) [0x560cae933bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x560caf088918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x560caef44c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x560caef47318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x560caef5c2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x560caef5d2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x560caef8c8e1]
9: (OSD::init()+0x61f) [0x560caea19daf]
10: (main()+0x3938) [0x560cae981fb8]
11: (__libc_start_main()+0xea) [0x7f0eab2ebcca]
12: (_start()+0x2e) [0x560cae99705e]

2020-07-31T16:18:45.255+0300 7f0eaadc9e00 -1 *** Caught signal (Aborted) **
in thread 7f0eaadc9e00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f0eab82e140]
2: (gsignal()+0x141) [0x7f0eab300cb1]
3: (abort()+0x123) [0x7f0eab2ea537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x560cae933a12]
5: (()+0x9e1bbf) [0x560cae933bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x560caf088918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x560caef44c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x560caef47318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x560caef5c2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x560caef5d2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x560caef8c8e1]
12: (OSD::init()+0x61f) [0x560caea19daf]
13: (main()+0x3938) [0x560cae981fb8]
14: (__libc_start_main()+0xea) [0x7f0eab2ebcca]
15: (_start()+0x2e) [0x560cae99705e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-318> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command assert hook 0x560cb9fb6570
-317> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command abort hook 0x560cb9fb6570
-316> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command leak_some_memory hook 0x560cb9fb6570
-315> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perfcounters_dump hook 0x560cb9fb6570
-314> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command 1 hook 0x560cb9fb6570
-313> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perf dump hook 0x560cb9fb6570
-312> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perfcounters_schema hook 0x560cb9fb6570
-311> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perf histogram dump hook 0x560cb9fb6570
-310> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command 2 hook 0x560cb9fb6570
-309> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perf schema hook 0x560cb9fb6570
-308> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perf histogram schema hook 0x560cb9fb6570
-307> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command perf reset hook 0x560cb9fb6570
-306> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config show hook 0x560cb9fb6570
-305> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config help hook 0x560cb9fb6570
-304> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config set hook 0x560cb9fb6570
-303> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config unset hook 0x560cb9fb6570
-302> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config get hook 0x560cb9fb6570
-301> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config diff hook 0x560cb9fb6570
-300> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command config diff get hook 0x560cb9fb6570
-299> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command injectargs hook 0x560cb9fb6570
-298> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command log flush hook 0x560cb9fb6570
-297> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command log dump hook 0x560cb9fb6570
-296> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command log reopen hook 0x560cb9fb6570
-295> 2020-07-31T16:18:19.655+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command dump_mempools hook 0x560cbac2c068
-294> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient: get_monmap_and_config
-293> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient: build_initial_monmap
-292> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:18:19.662763+0300
created 2020-07-31T16:18:19.662763+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-291> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding auth protocol: cephx
-290> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding auth protocol: cephx
-289> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding auth protocol: cephx
-288> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-287> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-286> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-285> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-284> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-283> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-282> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-281> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-280> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-279> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-278> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-277> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-276> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-275> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient: init
-274> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding auth protocol: cephx
-273> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding auth protocol: cephx
-272> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding auth protocol: cephx
-271> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: secure
-270> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: crc
-269> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: secure
-268> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: crc
-267> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: secure
-266> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: crc
-265> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: crc
-264> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: secure
-263> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: crc
-262> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: secure
-261> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: crc
-260> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 5 AuthRegistry(0x7ffccafcd430) adding con mode: secure
-259> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-258> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-257> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient: _reopen_session rank -1
-256> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient: _add_conns ranks=[12,11,13,1,7,10,9,3,5,2,8,4,0,6]
-255> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): picked mon.noname-b con 0x560cba056c00 addr v2:192.168.5.21:3300/0
-254> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): picked mon.noname-a con 0x560cba057000 addr v2:192.168.5.20:3300/0
-253> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): picked mon.noname-c con 0x560cba057400 addr v2:192.168.5.22:3300/0
-252> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): start opening mon connection
-251> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): start opening mon connection
-250> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): start opening mon connection
-249> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): _renew_subs
-248> 2020-07-31T16:18:19.659+0300 7f0eaadc9e00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:23:19.663792+0300
-247> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): get_auth_request con 0x560cba057400 auth_method 0
-246> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-245> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): _init_auth method 2
-244> 2020-07-31T16:18:19.659+0300 7f0ea9d6d700 10 monclient(hunting): get_auth_request con 0x560cba056c00 auth_method 0
-243> 2020-07-31T16:18:19.659+0300 7f0ea9d6d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-242> 2020-07-31T16:18:19.659+0300 7f0ea9d6d700 10 monclient(hunting): _init_auth method 2
-241> 2020-07-31T16:18:19.659+0300 7f0ea956c700 10 monclient(hunting): get_auth_request con 0x560cba057000 auth_method 0
-240> 2020-07-31T16:18:19.659+0300 7f0ea956c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-239> 2020-07-31T16:18:19.659+0300 7f0ea956c700 10 monclient(hunting): _init_auth method 2
-238> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): handle_auth_reply_more payload 9
-237> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-236> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-235> 2020-07-31T16:18:19.659+0300 7f0ea9d6d700 10 monclient(hunting): handle_auth_reply_more payload 9
-234> 2020-07-31T16:18:19.659+0300 7f0ea9d6d700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-233> 2020-07-31T16:18:19.659+0300 7f0ea9d6d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-232> 2020-07-31T16:18:19.659+0300 7f0ea956c700 10 monclient(hunting): handle_auth_reply_more payload 9
-231> 2020-07-31T16:18:19.659+0300 7f0ea956c700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-230> 2020-07-31T16:18:19.659+0300 7f0ea956c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-229> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient(hunting): handle_auth_done global_id 12842443 payload 450
-228> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient: _finish_hunting 0
-227> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 1 monclient: found mon.noname-c
-226> 2020-07-31T16:18:19.659+0300 7f0eaa56e700 10 monclient: _send_mon_message to mon.noname-c at v2:192.168.5.22:3300/0
-225> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: handle_monmap mon_map magic: 0 v1
-224> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: got monmap 28 from mon.noname-c (according to old e28)
-223> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-222> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: _finish_auth 0
-221> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:17:49.665222+0300)
-220> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: _send_mon_message to mon.ivy3 at v2:192.168.5.22:3300/0
-219> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: handle_config config(0 keys) v1
-218> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2020-07-31T16:18:19.663+0300 7f0ea7568700 4 set_mon_vals no callback set
-216> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: got monmap 28 from mon.ivy3 (according to old e28)
-215> 2020-07-31T16:18:19.663+0300 7f0ea8d6b700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-214> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 monclient: authenticate success, global_id 12842443
-213> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 10 monclient: get_monmap_and_config success
-212> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 10 monclient: shutdown
-211> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1171275
-209> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command 0 hook 0x560cb9fb42d0
-205> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command version hook 0x560cb9fb42d0
-204> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command git_version hook 0x560cb9fb42d0
-203> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command help hook 0x560cb9fb64c0
-202> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command get_command_descriptions hook 0x560cb9fb6030
-201> 2020-07-31T16:18:19.663+0300 7f0ea856a700 5 asok(0x560cba058000) entry start
-200> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev(0x560cbac36380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev(0x560cbac36380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore bluefs available hook 0x560cb9fb6070
-196> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore bluefs stats hook 0x560cb9fb6070
-195> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:18:19.663+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:18:19.667+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:18:19.667+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:18:19.667+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:18:19.667+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:18:19.987+0300 7f0eaadc9e00 1 bdev(0x560cbac36380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding auth protocol: cephx
-187> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding auth protocol: cephx
-186> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding auth protocol: cephx
-185> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-184> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-183> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-182> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-181> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-180> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-179> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-178> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-177> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-176> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-175> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: crc
-174> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba044940) adding con mode: secure
-173> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding auth protocol: cephx
-171> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding auth protocol: cephx
-170> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding auth protocol: cephx
-169> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: secure
-168> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: crc
-167> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: secure
-166> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: crc
-165> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: secure
-164> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: crc
-163> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: crc
-162> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: secure
-161> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: crc
-160> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: secure
-159> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: crc
-158> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba045940) adding con mode: secure
-157> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding auth protocol: cephx
-155> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding auth protocol: cephx
-154> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding auth protocol: cephx
-153> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: secure
-152> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: crc
-151> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: secure
-150> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: crc
-149> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: secure
-148> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: crc
-147> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: crc
-146> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: secure
-145> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: crc
-144> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: secure
-143> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: crc
-142> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba046940) adding con mode: secure
-141> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding auth protocol: cephx
-139> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding auth protocol: cephx
-138> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding auth protocol: cephx
-137> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: secure
-136> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: crc
-135> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: secure
-134> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: crc
-133> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: secure
-132> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: crc
-131> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: crc
-130> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: secure
-129> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: crc
-128> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: secure
-127> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: crc
-126> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cba047940) adding con mode: secure
-125> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding auth protocol: cephx
-123> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding auth protocol: cephx
-122> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding auth protocol: cephx
-121> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: secure
-120> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: crc
-119> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: secure
-118> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: crc
-117> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: secure
-116> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: crc
-115> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: crc
-114> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: secure
-113> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: crc
-112> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: secure
-111> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: crc
-110> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0c940) adding con mode: secure
-109> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding auth protocol: cephx
-107> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding auth protocol: cephx
-106> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding auth protocol: cephx
-105> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: secure
-104> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: crc
-103> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: secure
-102> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: crc
-101> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: secure
-100> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: crc
-99> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: crc
-98> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: secure
-97> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: crc
-96> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: secure
-95> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: crc
-94> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0d940) adding con mode: secure
-93> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding auth protocol: cephx
-91> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding auth protocol: cephx
-90> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding auth protocol: cephx
-89> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: secure
-88> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: crc
-87> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: secure
-86> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: crc
-85> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: secure
-84> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: crc
-83> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: crc
-82> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: secure
-81> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: crc
-80> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: secure
-79> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: crc
-78> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 5 AuthRegistry(0x560cbad0e940) adding con mode: secure
-77> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:18:20.215+0300 7f0eaadc9e00 -1 Falling back to public interface
-74> 2020-07-31T16:18:20.219+0300 7f0eaadc9e00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:18:20.219+0300 7f0eaadc9e00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:18:20.221355+0300
created 2020-07-31T16:18:20.221355+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:18:20.223+0300 7f0eaadc9e00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:18:20.223+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command objecter_requests hook 0x560cb9fb6930
-66> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac37180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore bluefs available hook 0x560cb9fb6950
-60> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore bluefs stats hook 0x560cb9fb6950
-59> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator dump bluefs-db hook 0x560cbad29500
-55> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator score bluefs-db hook 0x560cbad29500
-54> 2020-07-31T16:18:20.547+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator fragmentation bluefs-db hook 0x560cbad29500
-53> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:18:20.615+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 1 get compressor snappy = 0
-38> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 1 load compressor snappy
-37> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 1 add compressor snappy 0x560cbad29da0
-36> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 1 get compressor snappy = 0x560cbad29da0
-35> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 1 get compressor snappy = 0x560cbad29da0
-33> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator dump block hook 0x560cbb274f90
-32> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator score block hook 0x560cbb274f90
-31> 2020-07-31T16:18:20.639+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator fragmentation block hook 0x560cbb274f90
-30> 2020-07-31T16:18:21.059+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x560cbad299b0 EEXIST
-29> 2020-07-31T16:18:45.043+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:18:45.043+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:18:45.043+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:18:45.043+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:18:45.043+0300 7f0eaadc9e00 5 asok(0x560cba058000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:18:45.043+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore bluefs available hook 0x560cb9fb6950
-22> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore bluefs stats hook 0x560cb9fb6950
-21> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 1 bdev(0x560cbac36a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator dump bluefs-db hook 0x560cbad29470
-17> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator score bluefs-db hook 0x560cbad29470
-16> 2020-07-31T16:18:45.123+0300 7f0eaadc9e00 5 asok(0x560cba058000) register_command bluestore allocator fragmentation bluefs-db hook 0x560cbad29470
-15> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:18:45.175+0300 7f0eaadc9e00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:18:45.247+0300 7f0eaadc9e00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f0eaadc9e00 time 2020-07-31T16:18:45.247918+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x560cae9339b7]
2: (()+0x9e1bbf) [0x560cae933bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x560caf088918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x560caef44c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x560caef47318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x560caef5c2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x560caef5d2b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x560caef8c8e1]
9: (OSD::init()+0x61f) [0x560caea19daf]
10: (main()+0x3938) [0x560cae981fb8]
11: (__libc_start_main()+0xea) [0x7f0eab2ebcca]
12: (_start()+0x2e) [0x560cae99705e]

0> 2020-07-31T16:18:45.255+0300 7f0eaadc9e00 -1 *** Caught signal (Aborted) **
in thread 7f0eaadc9e00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f0eab82e140]
2: (gsignal()+0x141) [0x7f0eab300cb1]
3: (abort()+0x123) [0x7f0eab2ea537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x560cae933a12]
5: (()+0x9e1bbf) [0x560cae933bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x560caf088918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x560caef44c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x560caef47318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x560caef5c2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x560caef5d2b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x560caef8c8e1]
12: (OSD::init()+0x61f) [0x560caea19daf]
13: (main()+0x3938) [0x560cae981fb8]
14: (__libc_start_main()+0xea) [0x7f0eab2ebcca]
15: (_start()+0x2e) [0x560cae99705e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f0ea7568700 / signal_handler
7f0ea856a700 / admin_socket
7f0ea8d6b700 / service
7f0ea956c700 / msgr-worker-2
7f0ea9d6d700 / msgr-worker-1
7f0eaa56e700 / msgr-worker-0
7f0eaadc9e00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:19:15.839+0300 7febe921ce00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:19:15.839+0300 7febe921ce00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1171720
2020-07-31T16:19:15.839+0300 7febe921ce00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728c380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728c380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:16.155+0300 7febe921ce00 1 bdev(0x55ca4728c380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:16.407+0300 7febe921ce00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:19:16.407+0300 7febe921ce00 -1 Falling back to public interface
2020-07-31T16:19:16.411+0300 7febe921ce00 0 load: jerasure load: lrc load: isa
2020-07-31T16:19:16.411+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:16.411+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:16.411+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:16.415+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:16.719+0300 7febe921ce00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:19:16.719+0300 7febe921ce00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:19:40.455+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:40.571+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:40.571+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:40.575+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:19:40.683+0300 7febe921ce00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7febe921ce00 time 2020-07-31T16:19:40.682892+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x55ca3ad269b7]
2: (()+0x9e1bbf) [0x55ca3ad26bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55ca3b47b918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55ca3b337c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x55ca3b33a318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x55ca3b34f2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x55ca3b3502b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x55ca3b37f8e1]
9: (OSD::init()+0x61f) [0x55ca3ae0cdaf]
10: (main()+0x3938) [0x55ca3ad74fb8]
11: (__libc_start_main()+0xea) [0x7febe973ecca]
12: (_start()+0x2e) [0x55ca3ad8a05e]

2020-07-31T16:19:40.691+0300 7febe921ce00 -1 *** Caught signal (Aborted) **
in thread 7febe921ce00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7febe9c81140]
2: (gsignal()+0x141) [0x7febe9753cb1]
3: (abort()+0x123) [0x7febe973d537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x55ca3ad26a12]
5: (()+0x9e1bbf) [0x55ca3ad26bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55ca3b47b918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55ca3b337c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x55ca3b33a318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x55ca3b34f2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x55ca3b3502b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x55ca3b37f8e1]
12: (OSD::init()+0x61f) [0x55ca3ae0cdaf]
13: (main()+0x3938) [0x55ca3ad74fb8]
14: (__libc_start_main()+0xea) [0x7febe973ecca]
15: (_start()+0x2e) [0x55ca3ad8a05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-302> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command assert hook 0x55ca4660c570
-301> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command abort hook 0x55ca4660c570
-300> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command leak_some_memory hook 0x55ca4660c570
-299> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perfcounters_dump hook 0x55ca4660c570
-298> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command 1 hook 0x55ca4660c570
-297> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perf dump hook 0x55ca4660c570
-296> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perfcounters_schema hook 0x55ca4660c570
-295> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perf histogram dump hook 0x55ca4660c570
-294> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command 2 hook 0x55ca4660c570
-293> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perf schema hook 0x55ca4660c570
-292> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perf histogram schema hook 0x55ca4660c570
-291> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command perf reset hook 0x55ca4660c570
-290> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config show hook 0x55ca4660c570
-289> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config help hook 0x55ca4660c570
-288> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config set hook 0x55ca4660c570
-287> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config unset hook 0x55ca4660c570
-286> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config get hook 0x55ca4660c570
-285> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config diff hook 0x55ca4660c570
-284> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command config diff get hook 0x55ca4660c570
-283> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command injectargs hook 0x55ca4660c570
-282> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command log flush hook 0x55ca4660c570
-281> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command log dump hook 0x55ca4660c570
-280> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command log reopen hook 0x55ca4660c570
-279> 2020-07-31T16:19:15.831+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command dump_mempools hook 0x55ca47282068
-278> 2020-07-31T16:19:15.835+0300 7febe921ce00 10 monclient: get_monmap_and_config
-277> 2020-07-31T16:19:15.835+0300 7febe921ce00 10 monclient: build_initial_monmap
-276> 2020-07-31T16:19:15.835+0300 7febe921ce00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:19:15.840216+0300
created 2020-07-31T16:19:15.840216+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-275> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding auth protocol: cephx
-274> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding auth protocol: cephx
-273> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding auth protocol: cephx
-272> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-271> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-270> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-269> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-268> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-267> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-266> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-265> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-264> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-263> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-262> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-261> 2020-07-31T16:19:15.835+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-260> 2020-07-31T16:19:15.835+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-259> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient: init
-258> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding auth protocol: cephx
-257> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding auth protocol: cephx
-256> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding auth protocol: cephx
-255> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: secure
-254> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: crc
-253> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: secure
-252> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: crc
-251> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: secure
-250> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: crc
-249> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: crc
-248> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: secure
-247> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: crc
-246> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: secure
-245> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: crc
-244> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 AuthRegistry(0x7ffd84cb2760) adding con mode: secure
-243> 2020-07-31T16:19:15.839+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-242> 2020-07-31T16:19:15.839+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-241> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient: _reopen_session rank -1
-240> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient: _add_conns ranks=[11,2,4,3,10,9,13,0,7,5,6,8,12,1]
-239> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient(hunting): picked mon.noname-a con 0x55ca466acc00 addr v2:192.168.5.20:3300/0
-238> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient(hunting): picked mon.noname-f-legacy con 0x55ca466ad000 addr v1:192.168.5.15:6789/0
-237> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient(hunting): picked mon.noname-a-legacy con 0x55ca466ad400 addr v1:192.168.5.20:6789/0
-236> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient(hunting): start opening mon connection
-235> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient(hunting): _renew_subs
-234> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:24:15.841490+0300
-233> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): get_auth_request con 0x55ca466acc00 auth_method 0
-232> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-231> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): _init_auth method 2
-230> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): handle_auth_reply_more payload 9
-229> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-228> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-227> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient(hunting): handle_auth_done global_id 12838552 payload 450
-226> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient: _finish_hunting 0
-225> 2020-07-31T16:19:15.839+0300 7febe81c0700 1 monclient: found mon.noname-a
-224> 2020-07-31T16:19:15.839+0300 7febe81c0700 10 monclient: _send_mon_message to mon.noname-a at v2:192.168.5.20:3300/0
-223> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: handle_monmap mon_map magic: 0 v1
-222> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: got monmap 28 from mon.noname-a (according to old e28)
-221> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-220> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: _finish_auth 0
-219> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:18:45.842723+0300)
-218> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: _send_mon_message to mon.ivy1 at v2:192.168.5.20:3300/0
-217> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 monclient: authenticate success, global_id 12838552
-216> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: handle_config config(0 keys) v1
-215> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient: get_monmap_and_config success
-214> 2020-07-31T16:19:15.839+0300 7febe921ce00 10 monclient: shutdown
-213> 2020-07-31T16:19:15.839+0300 7febe71be700 10 monclient: discarding stray monitor message mon_map magic: 0 v1
-212> 2020-07-31T16:19:15.839+0300 7febe59bb700 4 set_mon_vals no callback set
-211> 2020-07-31T16:19:15.839+0300 7febe921ce00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:19:15.839+0300 7febe921ce00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1171720
-209> 2020-07-31T16:19:15.839+0300 7febe921ce00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command 0 hook 0x55ca4660a2d0
-205> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command version hook 0x55ca4660a2d0
-204> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command git_version hook 0x55ca4660a2d0
-203> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command help hook 0x55ca4660c4b0
-202> 2020-07-31T16:19:15.839+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command get_command_descriptions hook 0x55ca4660c030
-201> 2020-07-31T16:19:15.839+0300 7febe69bd700 5 asok(0x55ca466ae000) entry start
-200> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728c380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728c380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:19:15.843+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore bluefs available hook 0x55ca4660c070
-196> 2020-07-31T16:19:15.843+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore bluefs stats hook 0x55ca4660c070
-195> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:19:15.843+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:19:15.843+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:19:15.843+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:19:16.155+0300 7febe921ce00 1 bdev(0x55ca4728c380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding auth protocol: cephx
-187> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding auth protocol: cephx
-186> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding auth protocol: cephx
-185> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: secure
-184> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: crc
-183> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: secure
-182> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: crc
-181> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: secure
-180> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: crc
-179> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: crc
-178> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: secure
-177> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: crc
-176> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: secure
-175> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: crc
-174> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669c940) adding con mode: secure
-173> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding auth protocol: cephx
-171> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding auth protocol: cephx
-170> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding auth protocol: cephx
-169> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: secure
-168> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: crc
-167> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: secure
-166> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: crc
-165> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: secure
-164> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: crc
-163> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: crc
-162> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: secure
-161> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: crc
-160> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: secure
-159> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: crc
-158> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669b940) adding con mode: secure
-157> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding auth protocol: cephx
-155> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding auth protocol: cephx
-154> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding auth protocol: cephx
-153> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-152> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-151> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-150> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-149> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-148> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-147> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-146> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-145> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-144> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-143> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: crc
-142> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669a940) adding con mode: secure
-141> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding auth protocol: cephx
-139> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding auth protocol: cephx
-138> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding auth protocol: cephx
-137> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: secure
-136> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: crc
-135> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: secure
-134> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: crc
-133> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: secure
-132> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: crc
-131> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: crc
-130> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: secure
-129> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: crc
-128> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: secure
-127> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: crc
-126> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4669d940) adding con mode: secure
-125> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding auth protocol: cephx
-123> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding auth protocol: cephx
-122> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding auth protocol: cephx
-121> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: secure
-120> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: crc
-119> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: secure
-118> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: crc
-117> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: secure
-116> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: crc
-115> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: crc
-114> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: secure
-113> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: crc
-112> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: secure
-111> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: crc
-110> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47348940) adding con mode: secure
-109> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding auth protocol: cephx
-107> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding auth protocol: cephx
-106> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding auth protocol: cephx
-105> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: secure
-104> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: crc
-103> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: secure
-102> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: crc
-101> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: secure
-100> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: crc
-99> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: crc
-98> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: secure
-97> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: crc
-96> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: secure
-95> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: crc
-94> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca47349940) adding con mode: secure
-93> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding auth protocol: cephx
-91> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding auth protocol: cephx
-90> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding auth protocol: cephx
-89> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: secure
-88> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: crc
-87> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: secure
-86> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: crc
-85> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: secure
-84> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: crc
-83> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: crc
-82> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: secure
-81> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: crc
-80> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: secure
-79> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: crc
-78> 2020-07-31T16:19:16.407+0300 7febe921ce00 5 AuthRegistry(0x55ca4734a940) adding con mode: secure
-77> 2020-07-31T16:19:16.407+0300 7febe921ce00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:19:16.407+0300 7febe921ce00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:19:16.407+0300 7febe921ce00 -1 Falling back to public interface
-74> 2020-07-31T16:19:16.407+0300 7febe921ce00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:19:16.407+0300 7febe921ce00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:19:16.412456+0300
created 2020-07-31T16:19:16.412456+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:19:16.411+0300 7febe921ce00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:19:16.411+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:19:16.411+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:19:16.411+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:19:16.415+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:19:16.719+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command objecter_requests hook 0x55ca4660c950
-66> 2020-07-31T16:19:16.719+0300 7febe921ce00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:19:16.719+0300 7febe921ce00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728d180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:19:16.719+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore bluefs available hook 0x55ca4660c970
-60> 2020-07-31T16:19:16.719+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore bluefs stats hook 0x55ca4660c970
-59> 2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:19:16.719+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:19:16.719+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator dump bluefs-db hook 0x55ca47361380
-55> 2020-07-31T16:19:16.719+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator score bluefs-db hook 0x55ca47361380
-54> 2020-07-31T16:19:16.719+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator fragmentation bluefs-db hook 0x55ca47361380
-53> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:19:16.775+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:19:16.799+0300 7febe921ce00 1 get compressor snappy = 0
-38> 2020-07-31T16:19:16.799+0300 7febe921ce00 1 load compressor snappy
-37> 2020-07-31T16:19:16.799+0300 7febe921ce00 1 add compressor snappy 0x55ca47361c20
-36> 2020-07-31T16:19:16.799+0300 7febe921ce00 1 get compressor snappy = 0x55ca47361c20
-35> 2020-07-31T16:19:16.799+0300 7febe921ce00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:19:16.799+0300 7febe921ce00 1 get compressor snappy = 0x55ca47361c20
-33> 2020-07-31T16:19:16.803+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator dump block hook 0x55ca478a8e70
-32> 2020-07-31T16:19:16.803+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator score block hook 0x55ca478a8e70
-31> 2020-07-31T16:19:16.803+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator fragmentation block hook 0x55ca478a8e70
-30> 2020-07-31T16:19:17.219+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x55ca478a9530 EEXIST
-29> 2020-07-31T16:19:40.455+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:19:40.455+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:19:40.455+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:19:40.455+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:19:40.455+0300 7febe921ce00 5 asok(0x55ca466ae000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:19:40.455+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:19:40.571+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore bluefs available hook 0x55ca4660c970
-22> 2020-07-31T16:19:40.571+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore bluefs stats hook 0x55ca4660c970
-21> 2020-07-31T16:19:40.571+0300 7febe921ce00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:19:40.571+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:19:40.575+0300 7febe921ce00 1 bdev(0x55ca4728ca80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:19:40.575+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator dump bluefs-db hook 0x55ca473612f0
-17> 2020-07-31T16:19:40.575+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator score bluefs-db hook 0x55ca473612f0
-16> 2020-07-31T16:19:40.575+0300 7febe921ce00 5 asok(0x55ca466ae000) register_command bluestore allocator fragmentation bluefs-db hook 0x55ca473612f0
-15> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:19:40.643+0300 7febe921ce00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:19:40.683+0300 7febe921ce00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7febe921ce00 time 2020-07-31T16:19:40.682892+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x55ca3ad269b7]
2: (()+0x9e1bbf) [0x55ca3ad26bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55ca3b47b918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55ca3b337c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x55ca3b33a318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x55ca3b34f2b5]
7: (BlueStore::_deferred_replay()+0x313) [0x55ca3b3502b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x55ca3b37f8e1]
9: (OSD::init()+0x61f) [0x55ca3ae0cdaf]
10: (main()+0x3938) [0x55ca3ad74fb8]
11: (__libc_start_main()+0xea) [0x7febe973ecca]
12: (_start()+0x2e) [0x55ca3ad8a05e]

0> 2020-07-31T16:19:40.691+0300 7febe921ce00 -1 *** Caught signal (Aborted) **
in thread 7febe921ce00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7febe9c81140]
2: (gsignal()+0x141) [0x7febe9753cb1]
3: (abort()+0x123) [0x7febe973d537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x55ca3ad26a12]
5: (()+0x9e1bbf) [0x55ca3ad26bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55ca3b47b918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55ca3b337c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x55ca3b33a318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x55ca3b34f2b5]
10: (BlueStore::_deferred_replay()+0x313) [0x55ca3b3502b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x55ca3b37f8e1]
12: (OSD::init()+0x61f) [0x55ca3ae0cdaf]
13: (main()+0x3938) [0x55ca3ad74fb8]
14: (__libc_start_main()+0xea) [0x7febe973ecca]
15: (_start()+0x2e) [0x55ca3ad8a05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7febe59bb700 / signal_handler
7febe69bd700 / admin_socket
7febe71be700 / service
7febe81c0700 / msgr-worker-1
7febe921ce00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1172085
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:51.215+0300 7f8a9e56ae00 1 bdev(0x555c54be2380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:51.459+0300 7f8a9e56ae00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:19:51.459+0300 7f8a9e56ae00 -1 Falling back to public interface
2020-07-31T16:19:51.467+0300 7f8a9e56ae00 0 load: jerasure load: lrc load: isa
2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:20:31.135+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:20:31.467+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:20:31.467+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:20:31.467+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:20:31.543+0300 7f8a9e56ae00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f8a9e56ae00 time 2020-07-31T16:20:31.539008+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x555c485889b7]
2: (()+0x9e1bbf) [0x555c48588bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x555c48cdd918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x555c48b99c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x555c48b9c318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x555c48bb12b5]
7: (BlueStore::_deferred_replay()+0x313) [0x555c48bb22b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x555c48be18e1]
9: (OSD::init()+0x61f) [0x555c4866edaf]
10: (main()+0x3938) [0x555c485d6fb8]
11: (__libc_start_main()+0xea) [0x7f8a9ea8ccca]
12: (_start()+0x2e) [0x555c485ec05e]

2020-07-31T16:22:20.919+0300 7f8a95261700 -1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) read_random stalled read 0x1aa7ab7f000~200000 (buffered) since 4408556.383652s, timeout is 5.000000s
2020-07-31T16:22:20.927+0300 7f8a9e56ae00 -1 *** Caught signal (Aborted) **
in thread 7f8a9e56ae00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f8a9efcf140]
2: (gsignal()+0x141) [0x7f8a9eaa1cb1]
3: (abort()+0x123) [0x7f8a9ea8b537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x555c48588a12]
5: (()+0x9e1bbf) [0x555c48588bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x555c48cdd918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x555c48b99c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x555c48b9c318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x555c48bb12b5]
10: (BlueStore::_deferred_replay()+0x313) [0x555c48bb22b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x555c48be18e1]
12: (OSD::init()+0x61f) [0x555c4866edaf]
13: (main()+0x3938) [0x555c485d6fb8]
14: (__libc_start_main()+0xea) [0x7f8a9ea8ccca]
15: (_start()+0x2e) [0x555c485ec05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-312> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command assert hook 0x555c53f62570
-311> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command abort hook 0x555c53f62570
-310> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command leak_some_memory hook 0x555c53f62570
-309> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perfcounters_dump hook 0x555c53f62570
-308> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command 1 hook 0x555c53f62570
-307> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perf dump hook 0x555c53f62570
-306> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perfcounters_schema hook 0x555c53f62570
-305> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perf histogram dump hook 0x555c53f62570
-304> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command 2 hook 0x555c53f62570
-303> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perf schema hook 0x555c53f62570
-302> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perf histogram schema hook 0x555c53f62570
-301> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command perf reset hook 0x555c53f62570
-300> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config show hook 0x555c53f62570
-299> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config help hook 0x555c53f62570
-298> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config set hook 0x555c53f62570
-297> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config unset hook 0x555c53f62570
-296> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config get hook 0x555c53f62570
-295> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config diff hook 0x555c53f62570
-294> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command config diff get hook 0x555c53f62570
-293> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command injectargs hook 0x555c53f62570
-292> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command log flush hook 0x555c53f62570
-291> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command log dump hook 0x555c53f62570
-290> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command log reopen hook 0x555c53f62570
-289> 2020-07-31T16:19:50.903+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command dump_mempools hook 0x555c54bd8068
-288> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient: get_monmap_and_config
-287> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient: build_initial_monmap
-286> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:19:50.910845+0300
created 2020-07-31T16:19:50.910845+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-285> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding auth protocol: cephx
-284> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding auth protocol: cephx
-283> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding auth protocol: cephx
-282> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-281> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-280> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-279> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-278> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-277> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-276> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-275> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-274> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-273> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-272> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-271> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-270> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-269> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient: init
-268> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding auth protocol: cephx
-267> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding auth protocol: cephx
-266> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding auth protocol: cephx
-265> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: secure
-264> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: crc
-263> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: secure
-262> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: crc
-261> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: secure
-260> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: crc
-259> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: crc
-258> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: secure
-257> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: crc
-256> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: secure
-255> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: crc
-254> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 5 AuthRegistry(0x7ffffb7a39b0) adding con mode: secure
-253> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-252> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-251> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient: _reopen_session rank -1
-250> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient: _add_conns ranks=[11,9,0,1,7,8,3,12,6,10,4,5,2,13]
-249> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): picked mon.noname-a con 0x555c54002c00 addr v2:192.168.5.20:3300/0
-248> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): picked mon.noname-f con 0x555c54003000 addr v2:192.168.5.15:3300/0
-247> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): picked mon.noname-d-legacy con 0x555c54003400 addr v1:192.168.5.11:6789/0
-246> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): start opening mon connection
-245> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): start opening mon connection
-244> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): _renew_subs
-243> 2020-07-31T16:19:50.907+0300 7f8a9e56ae00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:24:50.912004+0300
-242> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): get_auth_request con 0x555c54003000 auth_method 0
-241> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-240> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): _init_auth method 2
-239> 2020-07-31T16:19:50.907+0300 7f8a9d50e700 10 monclient(hunting): get_auth_request con 0x555c54002c00 auth_method 0
-238> 2020-07-31T16:19:50.907+0300 7f8a9d50e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-237> 2020-07-31T16:19:50.907+0300 7f8a9d50e700 10 monclient(hunting): _init_auth method 2
-236> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): handle_auth_reply_more payload 9
-235> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-234> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-233> 2020-07-31T16:19:50.907+0300 7f8a9d50e700 10 monclient(hunting): handle_auth_reply_more payload 9
-232> 2020-07-31T16:19:50.907+0300 7f8a9d50e700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-231> 2020-07-31T16:19:50.907+0300 7f8a9d50e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-230> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient(hunting): handle_auth_done global_id 12838561 payload 450
-229> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient: _finish_hunting 0
-228> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 1 monclient: found mon.noname-f
-227> 2020-07-31T16:19:50.907+0300 7f8a9cd0d700 10 monclient: _send_mon_message to mon.noname-f at v2:192.168.5.15:3300/0
-226> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: handle_monmap mon_map magic: 0 v1
-225> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: got monmap 28 from mon.noname-f (according to old e28)
-224> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-223> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: _finish_auth 0
-222> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:19:20.913051+0300)
-221> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: _send_mon_message to mon.ripper5 at v2:192.168.5.15:3300/0
-220> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: handle_config config(0 keys) v1
-219> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: handle_monmap mon_map magic: 0 v1
-218> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: got monmap 28 from mon.ripper5 (according to old e28)
-217> 2020-07-31T16:19:50.911+0300 7f8a9ad09700 4 set_mon_vals no callback set
-216> 2020-07-31T16:19:50.911+0300 7f8a9c50c700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-215> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 monclient: authenticate success, global_id 12838561
-214> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 10 monclient: get_monmap_and_config success
-213> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 10 monclient: shutdown
-212> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 0 set uid:gid to 64045:64045 (ceph:ceph)
-211> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1172085
-210> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 0 pidfile_write: ignore empty --pid-file
-209> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) init /var/run/ceph/ceph-osd.2.asok
-208> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command 0 hook 0x555c53f602d0
-206> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command version hook 0x555c53f602d0
-205> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command git_version hook 0x555c53f602d0
-204> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command help hook 0x555c53f62370
-203> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command get_command_descriptions hook 0x555c53f62030
-202> 2020-07-31T16:19:50.911+0300 7f8a9bd0b700 5 asok(0x555c54004000) entry start
-201> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-200> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-199> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-198> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore bluefs available hook 0x555c53f62070
-197> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore bluefs stats hook 0x555c53f62070
-196> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-195> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-194> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-193> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore bluefs available
-192> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore bluefs stats
-191> 2020-07-31T16:19:50.911+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) close
-190> 2020-07-31T16:19:51.215+0300 7f8a9e56ae00 1 bdev(0x555c54be2380 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding auth protocol: cephx
-188> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding auth protocol: cephx
-187> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding auth protocol: cephx
-186> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: secure
-185> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: crc
-184> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: secure
-183> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: crc
-182> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: secure
-181> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: crc
-180> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: crc
-179> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: secure
-178> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: crc
-177> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: secure
-176> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: crc
-175> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff2940) adding con mode: secure
-174> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-173> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding auth protocol: cephx
-172> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding auth protocol: cephx
-171> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding auth protocol: cephx
-170> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: secure
-169> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: crc
-168> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: secure
-167> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: crc
-166> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: secure
-165> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: crc
-164> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: crc
-163> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: secure
-162> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: crc
-161> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: secure
-160> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: crc
-159> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff1940) adding con mode: secure
-158> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-157> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding auth protocol: cephx
-156> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding auth protocol: cephx
-155> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding auth protocol: cephx
-154> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-153> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-152> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-151> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-150> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-149> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-148> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-147> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-146> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-145> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-144> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: crc
-143> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff0940) adding con mode: secure
-142> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-141> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding auth protocol: cephx
-140> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding auth protocol: cephx
-139> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding auth protocol: cephx
-138> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: secure
-137> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: crc
-136> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: secure
-135> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: crc
-134> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: secure
-133> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: crc
-132> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: crc
-131> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: secure
-130> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: crc
-129> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: secure
-128> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: crc
-127> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c53ff3940) adding con mode: secure
-126> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-125> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding auth protocol: cephx
-124> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding auth protocol: cephx
-123> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding auth protocol: cephx
-122> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: secure
-121> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: crc
-120> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: secure
-119> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: crc
-118> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: secure
-117> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: crc
-116> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: crc
-115> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: secure
-114> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: crc
-113> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: secure
-112> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: crc
-111> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb8940) adding con mode: secure
-110> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-109> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding auth protocol: cephx
-108> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding auth protocol: cephx
-107> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding auth protocol: cephx
-106> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: secure
-105> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: crc
-104> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: secure
-103> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: crc
-102> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: secure
-101> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: crc
-100> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: crc
-99> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: secure
-98> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: crc
-97> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: secure
-96> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: crc
-95> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cb9940) adding con mode: secure
-94> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-93> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding auth protocol: cephx
-92> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding auth protocol: cephx
-91> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding auth protocol: cephx
-90> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: secure
-89> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: crc
-88> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: secure
-87> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: crc
-86> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: secure
-85> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: crc
-84> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: crc
-83> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: secure
-82> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: crc
-81> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: secure
-80> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: crc
-79> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 5 AuthRegistry(0x555c54cba940) adding con mode: secure
-78> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-77> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-76> 2020-07-31T16:19:51.459+0300 7f8a9e56ae00 -1 Falling back to public interface
-75> 2020-07-31T16:19:51.463+0300 7f8a9e56ae00 10 monclient: build_initial_monmap
-74> 2020-07-31T16:19:51.463+0300 7f8a9e56ae00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:19:51.465859+0300
created 2020-07-31T16:19:51.465859+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-73> 2020-07-31T16:19:51.467+0300 7f8a9e56ae00 0 load: jerasure load: lrc load: isa
-72> 2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-71> 2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-70> 2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-69> 2020-07-31T16:19:51.467+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) close
-68> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command objecter_requests hook 0x555c53f62950
-67> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-66> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-64> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-63> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be3180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-62> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore bluefs available hook 0x555c53f62970
-61> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore bluefs stats hook 0x555c53f62970
-60> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-59> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-58> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-57> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator dump bluefs-db hook 0x555c54ccd470
-56> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator score bluefs-db hook 0x555c54ccd470
-55> 2020-07-31T16:19:51.759+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator fragmentation bluefs-db hook 0x555c54ccd470
-54> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
-53> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
-52> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
-51> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-50> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
-49> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-48> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
-47> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
-46> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
-45> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
-44> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-43> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
-42> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-41> 2020-07-31T16:19:51.819+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
-40> 2020-07-31T16:19:51.843+0300 7f8a9e56ae00 1 get compressor snappy = 0
-39> 2020-07-31T16:19:51.843+0300 7f8a9e56ae00 1 load compressor snappy
-38> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 1 add compressor snappy 0x555c54ccdc80
-37> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 1 get compressor snappy = 0x555c54ccdc80
-36> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 1 load: compressor snappy loaded and registered
-35> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 1 get compressor snappy = 0x555c54ccdc80
-34> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator dump block hook 0x555c551077a0
-33> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator score block hook 0x555c551077a0
-32> 2020-07-31T16:19:51.847+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator fragmentation block hook 0x555c551077a0
-31> 2020-07-31T16:19:52.283+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x555c55107080 EEXIST
-30> 2020-07-31T16:20:31.135+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore allocator dump bluefs-db
-29> 2020-07-31T16:20:31.135+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore allocator fragmentation bluefs-db
-28> 2020-07-31T16:20:31.135+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore allocator score bluefs-db
-27> 2020-07-31T16:20:31.135+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore bluefs available
-26> 2020-07-31T16:20:31.135+0300 7f8a9e56ae00 5 asok(0x555c54004000) unregister_commands bluestore bluefs stats
-25> 2020-07-31T16:20:31.135+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) close
-24> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore bluefs available hook 0x555c53f62970
-23> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore bluefs stats hook 0x555c53f62970
-22> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-21> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-20> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-19> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator dump bluefs-db hook 0x555c54ccd3e0
-18> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator score bluefs-db hook 0x555c54ccd3e0
-17> 2020-07-31T16:20:31.467+0300 7f8a9e56ae00 5 asok(0x555c54004000) register_command bluestore allocator fragmentation bluefs-db hook 0x555c54ccd3e0
-16> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
-15> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
-14> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
-13> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-12> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
-11> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-10> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
-9> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compaction_readahead_size = 2097152
-8> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option compression = kNoCompression
-7> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option max_write_buffer_number = 32
-6> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-5> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option recycle_log_file_num = 32
-4> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option writable_file_max_buffer_size = 0
-3> 2020-07-31T16:20:31.499+0300 7f8a9e56ae00 0 set rocksdb option write_buffer_size = 33554432
-2> 2020-07-31T16:20:31.543+0300 7f8a9e56ae00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f8a9e56ae00 time 2020-07-31T16:20:31.539008+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x555c485889b7]
2: (()+0x9e1bbf) [0x555c48588bbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x555c48cdd918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x555c48b99c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x555c48b9c318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x555c48bb12b5]
7: (BlueStore::_deferred_replay()+0x313) [0x555c48bb22b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x555c48be18e1]
9: (OSD::init()+0x61f) [0x555c4866edaf]
10: (main()+0x3938) [0x555c485d6fb8]
11: (__libc_start_main()+0xea) [0x7f8a9ea8ccca]
12: (_start()+0x2e) [0x555c485ec05e]

-1> 2020-07-31T16:22:20.919+0300 7f8a95261700 -1 bdev(0x555c54be2a80 /var/lib/ceph/osd/ceph-2/block) read_random stalled read 0x1aa7ab7f000~200000 (buffered) since 4408556.383652s, timeout is 5.000000s
0> 2020-07-31T16:22:20.927+0300 7f8a9e56ae00 -1 *** Caught signal (Aborted) **
in thread 7f8a9e56ae00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f8a9efcf140]
2: (gsignal()+0x141) [0x7f8a9eaa1cb1]
3: (abort()+0x123) [0x7f8a9ea8b537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x555c48588a12]
5: (()+0x9e1bbf) [0x555c48588bbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x555c48cdd918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x555c48b99c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x555c48b9c318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x555c48bb12b5]
10: (BlueStore::_deferred_replay()+0x313) [0x555c48bb22b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x555c48be18e1]
12: (OSD::init()+0x61f) [0x555c4866edaf]
13: (main()+0x3938) [0x555c485d6fb8]
14: (__libc_start_main()+0xea) [0x7f8a9ea8ccca]
15: (_start()+0x2e) [0x555c485ec05e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f8a95261700 / rocksdb:low0
7f8a9ad09700 / signal_handler
7f8a9bd0b700 / admin_socket
7f8a9c50c700 / service
7f8a9cd0d700 / msgr-worker-2
7f8a9d50e700 / msgr-worker-1
7f8a9e56ae00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
2020-07-31T16:29:34.925+0300 7f447caade00 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-07-31T16:29:34.925+0300 7f447caade00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1174638
2020-07-31T16:29:34.925+0300 7f447caade00 0 pidfile_write: ignore empty --pid-file
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:29:35.233+0300 7f447caade00 1 bdev(0x55ae0c060380 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:29:35.473+0300 7f447caade00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
2020-07-31T16:29:35.473+0300 7f447caade00 -1 Falling back to public interface
2020-07-31T16:29:35.477+0300 7f447caade00 0 load: jerasure load: lrc load: isa
2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:29:35.797+0300 7f447caade00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:29:35.797+0300 7f447caade00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:31:28.773+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) close
2020-07-31T16:31:29.057+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
2020-07-31T16:31:29.057+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
2020-07-31T16:31:29.057+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
2020-07-31T16:31:29.177+0300 7f447caade00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f447caade00 time 2020-07-31T16:31:29.175567+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x55adff81f9b7]
2: (()+0x9e1bbf) [0x55adff81fbbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55adfff74918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55adffe30c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x55adffe33318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x55adffe482b5]
7: (BlueStore::_deferred_replay()+0x313) [0x55adffe492b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x55adffe788e1]
9: (OSD::init()+0x61f) [0x55adff905daf]
10: (main()+0x3938) [0x55adff86dfb8]
11: (__libc_start_main()+0xea) [0x7f447cfcfcca]
12: (_start()+0x2e) [0x55adff88305e]

2020-07-31T16:32:39.929+0300 7f447caade00 -1 *** Caught signal (Aborted) **
in thread 7f447caade00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f447d512140]
2: (gsignal()+0x141) [0x7f447cfe4cb1]
3: (abort()+0x123) [0x7f447cfce537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x55adff81fa12]
5: (()+0x9e1bbf) [0x55adff81fbbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55adfff74918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55adffe30c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x55adffe33318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x55adffe482b5]
10: (BlueStore::_deferred_replay()+0x313) [0x55adffe492b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x55adffe788e1]
12: (OSD::init()+0x61f) [0x55adff905daf]
13: (main()+0x3938) [0x55adff86dfb8]
14: (__libc_start_main()+0xea) [0x7f447cfcfcca]
15: (_start()+0x2e) [0x55adff88305e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-308> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command assert hook 0x55ae0b3e0570
-307> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command abort hook 0x55ae0b3e0570
-306> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command leak_some_memory hook 0x55ae0b3e0570
-305> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perfcounters_dump hook 0x55ae0b3e0570
-304> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command 1 hook 0x55ae0b3e0570
-303> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perf dump hook 0x55ae0b3e0570
-302> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perfcounters_schema hook 0x55ae0b3e0570
-301> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perf histogram dump hook 0x55ae0b3e0570
-300> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command 2 hook 0x55ae0b3e0570
-299> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perf schema hook 0x55ae0b3e0570
-298> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perf histogram schema hook 0x55ae0b3e0570
-297> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command perf reset hook 0x55ae0b3e0570
-296> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config show hook 0x55ae0b3e0570
-295> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config help hook 0x55ae0b3e0570
-294> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config set hook 0x55ae0b3e0570
-293> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config unset hook 0x55ae0b3e0570
-292> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config get hook 0x55ae0b3e0570
-291> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config diff hook 0x55ae0b3e0570
-290> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command config diff get hook 0x55ae0b3e0570
-289> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command injectargs hook 0x55ae0b3e0570
-288> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command log flush hook 0x55ae0b3e0570
-287> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command log dump hook 0x55ae0b3e0570
-286> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command log reopen hook 0x55ae0b3e0570
-285> 2020-07-31T16:29:34.917+0300 7f447caade00 5 asok(0x55ae0b482000) register_command dump_mempools hook 0x55ae0c056068
-284> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: get_monmap_and_config
-283> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: build_initial_monmap
-282> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:29:34.925961+0300
created 2020-07-31T16:29:34.925961+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-281> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding auth protocol: cephx
-280> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding auth protocol: cephx
-279> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding auth protocol: cephx
-278> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-277> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-276> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-275> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-274> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-273> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-272> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-271> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-270> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-269> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-268> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-267> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-266> 2020-07-31T16:29:34.921+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-265> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: init
-264> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding auth protocol: cephx
-263> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding auth protocol: cephx
-262> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding auth protocol: cephx
-261> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: secure
-260> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: crc
-259> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: secure
-258> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: crc
-257> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: secure
-256> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: crc
-255> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: crc
-254> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: secure
-253> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: crc
-252> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: secure
-251> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: crc
-250> 2020-07-31T16:29:34.921+0300 7f447caade00 5 AuthRegistry(0x7ffef434d6a0) adding con mode: secure
-249> 2020-07-31T16:29:34.921+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-248> 2020-07-31T16:29:34.921+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-247> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: _reopen_session rank -1
-246> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: _add_conns ranks=[1,3,8,13,9,7,12,2,0,6,10,11,4,5]
-245> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient(hunting): picked mon.noname-e-legacy con 0x55ae0b480c00 addr v1:192.168.5.14:6789/0
-244> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient(hunting): picked mon.noname-g-legacy con 0x55ae0b481000 addr v1:192.168.5.16:6789/0
-243> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient(hunting): picked mon.noname-e con 0x55ae0b481400 addr v2:192.168.5.14:3300/0
-242> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient(hunting): start opening mon connection
-241> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient(hunting): _renew_subs
-240> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient(hunting): authenticate will time out at 2020-07-31T16:34:34.927243+0300
-239> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): get_auth_request con 0x55ae0b481400 auth_method 0
-238> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-237> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): _init_auth method 2
-236> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): handle_auth_reply_more payload 9
-235> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-234> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-233> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient(hunting): _init_auth method 2
-232> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient(hunting): my global_id is 12845385
-231> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient(hunting): _init_auth method 2
-230> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient(hunting): my global_id is 12832920
-229> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient(hunting): handle_auth_done global_id 12845378 payload 450
-228> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient: _finish_hunting 0
-227> 2020-07-31T16:29:34.921+0300 7f447c252700 1 monclient: found mon.noname-e
-226> 2020-07-31T16:29:34.921+0300 7f447c252700 10 monclient: _send_mon_message to mon.noname-e at v2:192.168.5.14:3300/0
-225> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: handle_monmap mon_map magic: 0 v1
-224> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: got monmap 28 from mon.noname-e (according to old e28)
-223> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-222> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: _finish_auth 0
-221> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-07-31T16:29:04.928251+0300)
-220> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: _send_mon_message to mon.ripper4 at v2:192.168.5.14:3300/0
-219> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: handle_config config(0 keys) v1
-218> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: got monmap 28 from mon.ripper4 (according to old e28)
-216> 2020-07-31T16:29:34.921+0300 7f447aa4f700 10 monclient: dump:
epoch 28
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-24T13:05:00.613222+0300
created 2018-11-06T00:56:40.343651+0300
min_mon_release 15 (octopus)
0: [v2:192.168.5.14:3300/0,v1:192.168.5.14:6789/0] mon.ripper4
1: [v2:192.168.5.15:3300/0,v1:192.168.5.15:6789/0] mon.ripper5
2: [v2:192.168.5.16:3300/0,v1:192.168.5.16:6789/0] mon.ripper6
3: [v2:192.168.5.11:3300/0,v1:192.168.5.11:6789/0] mon.mature1
4: [v2:192.168.5.21:3300/0,v1:192.168.5.21:6789/0] mon.ivy2
5: [v2:192.168.5.22:3300/0,v1:192.168.5.22:6789/0] mon.ivy3
6: [v2:192.168.5.20:3300/0,v1:192.168.5.20:6789/0] mon.ivy1

-215> 2020-07-31T16:29:34.921+0300 7f447924c700 4 set_mon_vals no callback set
-214> 2020-07-31T16:29:34.921+0300 7f447caade00 5 monclient: authenticate success, global_id 12845378
-213> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: get_monmap_and_config success
-212> 2020-07-31T16:29:34.921+0300 7f447caade00 10 monclient: shutdown
-211> 2020-07-31T16:29:34.925+0300 7f447caade00 0 set uid:gid to 64045:64045 (ceph:ceph)
-210> 2020-07-31T16:29:34.925+0300 7f447caade00 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process ceph-osd, pid 1174638
-209> 2020-07-31T16:29:34.925+0300 7f447caade00 0 pidfile_write: ignore empty --pid-file
-208> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) init /var/run/ceph/ceph-osd.2.asok
-207> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-206> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command 0 hook 0x55ae0b3de2d0
-205> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command version hook 0x55ae0b3de2d0
-204> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command git_version hook 0x55ae0b3de2d0
-203> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command help hook 0x55ae0b3e0370
-202> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command get_command_descriptions hook 0x55ae0b3e0030
-201> 2020-07-31T16:29:34.925+0300 7f447a24e700 5 asok(0x55ae0b482000) entry start
-200> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-199> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060380 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-198> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060380 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-197> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore bluefs available hook 0x55ae0b3e0070
-196> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore bluefs stats hook 0x55ae0b3e0070
-195> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-194> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-193> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-192> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore bluefs available
-191> 2020-07-31T16:29:34.925+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore bluefs stats
-190> 2020-07-31T16:29:34.925+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) close
-189> 2020-07-31T16:29:35.233+0300 7f447caade00 1 bdev(0x55ae0c060380 /var/lib/ceph/osd/ceph-2/block) close
-188> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding auth protocol: cephx
-187> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding auth protocol: cephx
-186> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding auth protocol: cephx
-185> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: secure
-184> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: crc
-183> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: secure
-182> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: crc
-181> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: secure
-180> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: crc
-179> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: crc
-178> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: secure
-177> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: crc
-176> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: secure
-175> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: crc
-174> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b470940) adding con mode: secure
-173> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-172> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding auth protocol: cephx
-171> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding auth protocol: cephx
-170> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding auth protocol: cephx
-169> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: secure
-168> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: crc
-167> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: secure
-166> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: crc
-165> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: secure
-164> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: crc
-163> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: crc
-162> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: secure
-161> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: crc
-160> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: secure
-159> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: crc
-158> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46f940) adding con mode: secure
-157> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-156> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding auth protocol: cephx
-155> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding auth protocol: cephx
-154> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding auth protocol: cephx
-153> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-152> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-151> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-150> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-149> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-148> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-147> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-146> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-145> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-144> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-143> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: crc
-142> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b46e940) adding con mode: secure
-141> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-140> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding auth protocol: cephx
-139> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding auth protocol: cephx
-138> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding auth protocol: cephx
-137> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: secure
-136> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: crc
-135> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: secure
-134> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: crc
-133> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: secure
-132> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: crc
-131> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: crc
-130> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: secure
-129> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: crc
-128> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: secure
-127> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: crc
-126> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0b471940) adding con mode: secure
-125> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-124> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding auth protocol: cephx
-123> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding auth protocol: cephx
-122> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding auth protocol: cephx
-121> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: secure
-120> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: crc
-119> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: secure
-118> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: crc
-117> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: secure
-116> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: crc
-115> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: crc
-114> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: secure
-113> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: crc
-112> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: secure
-111> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: crc
-110> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c136940) adding con mode: secure
-109> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-108> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding auth protocol: cephx
-107> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding auth protocol: cephx
-106> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding auth protocol: cephx
-105> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: secure
-104> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: crc
-103> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: secure
-102> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: crc
-101> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: secure
-100> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: crc
-99> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: crc
-98> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: secure
-97> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: crc
-96> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: secure
-95> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: crc
-94> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c137940) adding con mode: secure
-93> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-92> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding auth protocol: cephx
-91> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding auth protocol: cephx
-90> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding auth protocol: cephx
-89> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: secure
-88> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: crc
-87> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: secure
-86> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: crc
-85> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: secure
-84> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: crc
-83> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: crc
-82> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: secure
-81> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: crc
-80> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: secure
-79> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: crc
-78> 2020-07-31T16:29:35.473+0300 7f447caade00 5 AuthRegistry(0x55ae0c138940) adding con mode: secure
-77> 2020-07-31T16:29:35.473+0300 7f447caade00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring
-76> 2020-07-31T16:29:35.473+0300 7f447caade00 0 starting osd.2 osd_data /var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
-75> 2020-07-31T16:29:35.473+0300 7f447caade00 -1 Falling back to public interface
-74> 2020-07-31T16:29:35.477+0300 7f447caade00 10 monclient: build_initial_monmap
-73> 2020-07-31T16:29:35.477+0300 7f447caade00 10 monclient: monmap:
epoch 0
fsid 3705b320-3505-4561-a3aa-5b0475aadf9d
last_changed 2020-07-31T16:29:35.481056+0300
created 2020-07-31T16:29:35.481056+0300
min_mon_release 0 (unknown)
0: v1:192.168.5.11:6789/0 mon.noname-d-legacy
1: v1:192.168.5.14:6789/0 mon.noname-e-legacy
2: v1:192.168.5.15:6789/0 mon.noname-f-legacy
3: v1:192.168.5.16:6789/0 mon.noname-g-legacy
4: v1:192.168.5.20:6789/0 mon.noname-a-legacy
5: v1:192.168.5.21:6789/0 mon.noname-b-legacy
6: v1:192.168.5.22:6789/0 mon.noname-c-legacy
7: v2:192.168.5.11:3300/0 mon.noname-d
8: v2:192.168.5.14:3300/0 mon.noname-e
9: v2:192.168.5.15:3300/0 mon.noname-f
10: v2:192.168.5.16:3300/0 mon.noname-g
11: v2:192.168.5.20:3300/0 mon.noname-a
12: v2:192.168.5.21:3300/0 mon.noname-b
13: v2:192.168.5.22:3300/0 mon.noname-c

-72> 2020-07-31T16:29:35.477+0300 7f447caade00 0 load: jerasure load: lrc load: isa
-71> 2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-70> 2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-69> 2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-68> 2020-07-31T16:29:35.481+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) close
-67> 2020-07-31T16:29:35.797+0300 7f447caade00 5 asok(0x55ae0b482000) register_command objecter_requests hook 0x55ae0b3e0940
-66> 2020-07-31T16:29:35.797+0300 7f447caade00 0 osd.2:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-65> 2020-07-31T16:29:35.797+0300 7f447caade00 0 osd.2:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
-64> 2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-63> 2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-62> 2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c061180 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-61> 2020-07-31T16:29:35.797+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore bluefs available hook 0x55ae0b3e0960
-60> 2020-07-31T16:29:35.797+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore bluefs stats hook 0x55ae0b3e0960
-59> 2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-58> 2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-57> 2020-07-31T16:29:35.797+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-56> 2020-07-31T16:29:35.797+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator dump bluefs-db hook 0x55ae0c14f3b0
-55> 2020-07-31T16:29:35.797+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator score bluefs-db hook 0x55ae0c14f3b0
-54> 2020-07-31T16:29:35.797+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator fragmentation bluefs-db hook 0x55ae0c14f3b0
-53> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
-52> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
-51> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
-50> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-49> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
-48> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
-47> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
-46> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
-45> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
-44> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
-43> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-42> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
-41> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
-40> 2020-07-31T16:29:35.849+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
-39> 2020-07-31T16:29:35.873+0300 7f447caade00 1 get compressor snappy = 0
-38> 2020-07-31T16:29:35.873+0300 7f447caade00 1 load compressor snappy
-37> 2020-07-31T16:29:35.873+0300 7f447caade00 1 add compressor snappy 0x55ae0c14fe30
-36> 2020-07-31T16:29:35.873+0300 7f447caade00 1 get compressor snappy = 0x55ae0c14fe30
-35> 2020-07-31T16:29:35.873+0300 7f447caade00 1 load: compressor snappy loaded and registered
-34> 2020-07-31T16:29:35.873+0300 7f447caade00 1 get compressor snappy = 0x55ae0c14fe30
-33> 2020-07-31T16:29:35.877+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator dump block hook 0x55ae0c6e0210
-32> 2020-07-31T16:29:35.877+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator score block hook 0x55ae0c6e0210
-31> 2020-07-31T16:29:35.877+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator fragmentation block hook 0x55ae0c6e0210
-30> 2020-07-31T16:29:36.293+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator dump block cmddesc bluestore allocator dump block hook 0x55ae0c6a0d50 EEXIST
-29> 2020-07-31T16:31:28.773+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore allocator dump bluefs-db
-28> 2020-07-31T16:31:28.773+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore allocator fragmentation bluefs-db
-27> 2020-07-31T16:31:28.773+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore allocator score bluefs-db
-26> 2020-07-31T16:31:28.773+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore bluefs available
-25> 2020-07-31T16:31:28.773+0300 7f447caade00 5 asok(0x55ae0b482000) unregister_commands bluestore bluefs stats
-24> 2020-07-31T16:31:28.773+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) close
-23> 2020-07-31T16:31:29.057+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore bluefs available hook 0x55ae0b3e0960
-22> 2020-07-31T16:31:29.057+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore bluefs stats hook 0x55ae0b3e0960
-21> 2020-07-31T16:31:29.057+0300 7f447caade00 1 bdev create path /var/lib/ceph/osd/ceph-2/block type kernel
-20> 2020-07-31T16:31:29.057+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block
-19> 2020-07-31T16:31:29.057+0300 7f447caade00 1 bdev(0x55ae0c060a80 /var/lib/ceph/osd/ceph-2/block) open size 3796751089664 (0x37400000000, 3.5 TiB) block_size 4096 (4 KiB) non-rotational discard supported
-18> 2020-07-31T16:31:29.057+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator dump bluefs-db hook 0x55ae0c14f320
-17> 2020-07-31T16:31:29.057+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator score bluefs-db hook 0x55ae0c14f320
-16> 2020-07-31T16:31:29.057+0300 7f447caade00 5 asok(0x55ae0b482000) register_command bluestore allocator fragmentation bluefs-db hook 0x55ae0c14f320
-15> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
-14> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
-13> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
-12> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-11> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
-10> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
-9> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
-8> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compaction_readahead_size = 2097152
-7> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option compression = kNoCompression
-6> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option max_write_buffer_number = 32
-5> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option min_write_buffer_number_to_merge = 8
-4> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option recycle_log_file_num = 32
-3> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option writable_file_max_buffer_size = 0
-2> 2020-07-31T16:31:29.105+0300 7f447caade00 0 set rocksdb option write_buffer_size = 33554432
-1> 2020-07-31T16:31:29.177+0300 7f447caade00 -1 /build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::aio_write(uint64_t, ceph::bufferlist&, IOContext*, bool, int)' thread 7f447caade00 time 2020-07-31T16:31:29.175567+0300
/build/ceph-15.2.4/src/os/bluestore/KernelDevice.cc: 892: FAILED ceph_assert(is_valid_io(off, len))

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x155) [0x55adff81f9b7]
2: (()+0x9e1bbf) [0x55adff81fbbf]
3: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55adfff74918]
4: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55adffe30c3d]
5: (BlueStore::deferred_try_submit()+0x428) [0x55adffe33318]
6: (BlueStore::_osr_drain_all()+0x1f5) [0x55adffe482b5]
7: (BlueStore::_deferred_replay()+0x313) [0x55adffe492b3]
8: (BlueStore::_mount(bool, bool)+0x611) [0x55adffe788e1]
9: (OSD::init()+0x61f) [0x55adff905daf]
10: (main()+0x3938) [0x55adff86dfb8]
11: (__libc_start_main()+0xea) [0x7f447cfcfcca]
12: (_start()+0x2e) [0x55adff88305e]

0> 2020-07-31T16:32:39.929+0300 7f447caade00 -1 *** Caught signal (Aborted) **
in thread 7f447caade00 thread_name:ceph-osd

ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)
1: (()+0x14140) [0x7f447d512140]
2: (gsignal()+0x141) [0x7f447cfe4cb1]
3: (abort()+0x123) [0x7f447cfce537]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x55adff81fa12]
5: (()+0x9e1bbf) [0x55adff81fbbf]
6: (KernelDevice::aio_write(unsigned long, ceph::buffer::v15_2_0::list&, IOContext*, bool, int)+0x98) [0x55adfff74918]
7: (BlueStore::_deferred_submit_unlock(BlueStore::OpSequencer*)+0x2fd) [0x55adffe30c3d]
8: (BlueStore::deferred_try_submit()+0x428) [0x55adffe33318]
9: (BlueStore::_osr_drain_all()+0x1f5) [0x55adffe482b5]
10: (BlueStore::_deferred_replay()+0x313) [0x55adffe492b3]
11: (BlueStore::_mount(bool, bool)+0x611) [0x55adffe788e1]
12: (OSD::init()+0x61f) [0x55adff905daf]
13: (main()+0x3938) [0x55adff86dfb8]
14: (__libc_start_main()+0xea) [0x7f447cfcfcca]
15: (_start()+0x2e) [0x55adff88305e]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
0/ 0 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 0 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 0 client
0/ 0 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 0 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
0/ 0 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
0/ 0 bluestore
0/ 0 bluefs
1/ 3 bdev
1/ 5 kstore
0/ 0 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7f447924c700 / signal_handler
7f447a24e700 / admin_socket
7f447aa4f700 / service
7f447c252700 / msgr-worker-0
7f447caade00 / ceph-osd
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
(1-1/2)