2021-02-02 12:53:18.084995 7f5da959dec0 0 set uid:gid to 167:167 (ceph:ceph) 2021-02-02 12:53:18.085017 7f5da959dec0 0 ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous (stable), process ceph-osd, pid 374299 2021-02-02 12:53:18.094069 7f5da959dec0 0 pidfile_write: ignore empty --pid-file 2021-02-02 12:53:18.138962 7f5da959dec0 0 load: jerasure load: lrc load: isa 2021-02-02 12:53:18.139055 7f5da959dec0 1 bdev create path /var/lib/ceph/osd/ceph-108/block type kernel 2021-02-02 12:53:18.139067 7f5da959dec0 1 bdev(0x55670aad9200 /var/lib/ceph/osd/ceph-108/block) open path /var/lib/ceph/osd/ceph-108/block 2021-02-02 12:53:18.139306 7f5da959dec0 1 bdev(0x55670aad9200 /var/lib/ceph/osd/ceph-108/block) open size 1000203091968 (0xe8e0c00000, 932GiB) block_size 4096 (4KiB) non-rotational 2021-02-02 12:53:18.139515 7f5da959dec0 1 bluestore(/var/lib/ceph/osd/ceph-108) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2 2021-02-02 12:53:18.139546 7f5da959dec0 1 bdev(0x55670aad9200 /var/lib/ceph/osd/ceph-108/block) close 2021-02-02 12:53:18.399641 7f5da959dec0 1 bluestore(/var/lib/ceph/osd/ceph-108) _mount path /var/lib/ceph/osd/ceph-108 2021-02-02 12:53:18.399991 7f5da959dec0 1 bdev create path /var/lib/ceph/osd/ceph-108/block type kernel 2021-02-02 12:53:18.400002 7f5da959dec0 1 bdev(0x55670aad8600 /var/lib/ceph/osd/ceph-108/block) open path /var/lib/ceph/osd/ceph-108/block 2021-02-02 12:53:18.400156 7f5da959dec0 1 bdev(0x55670aad8600 /var/lib/ceph/osd/ceph-108/block) open size 1000203091968 (0xe8e0c00000, 932GiB) block_size 4096 (4KiB) non-rotational 2021-02-02 12:53:18.400413 7f5da959dec0 1 bluestore(/var/lib/ceph/osd/ceph-108) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2 2021-02-02 12:53:18.400564 7f5da959dec0 1 bdev create path /var/lib/ceph/osd/ceph-108/block type kernel 2021-02-02 12:53:18.400571 7f5da959dec0 1 bdev(0x55670aad6600 /var/lib/ceph/osd/ceph-108/block) open path /var/lib/ceph/osd/ceph-108/block 2021-02-02 12:53:18.400716 7f5da959dec0 1 bdev(0x55670aad6600 /var/lib/ceph/osd/ceph-108/block) open size 1000203091968 (0xe8e0c00000, 932GiB) block_size 4096 (4KiB) non-rotational 2021-02-02 12:53:18.400731 7f5da959dec0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-108/block size 932GiB 2021-02-02 12:53:18.400758 7f5da959dec0 1 bluefs mount 2021-02-02 12:53:18.401032 7f5da959dec0 1 bluefs _init_alloc id 1 alloc_size 0x10000 size 0xe8e0c00000 2021-02-02 13:21:09.304110 7f5da959dec0 -1 *** Caught signal (Segmentation fault) ** in thread 7f5da959dec0 thread_name:ceph-osd ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous (stable) 1: (()+0xa76e51) [0x5566ff53ce51] 2: (()+0xf630) [0x7f5da6a40630] 3: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x452) [0x5566ff4efcd2] 4: (BlueFS::_replay(bool)+0x48d) [0x5566ff50378d] 5: (BlueFS::mount()+0x1d4) [0x5566ff507374] 6: (BlueStore::_open_db(bool)+0x1857) [0x5566ff415827] 7: (BlueStore::_mount(bool)+0x2f6) [0x5566ff44acf6] 8: (OSD::init()+0x472) [0x5566fefe9762] 9: (main()+0x2f49) [0x5566feeea719] 10: (__libc_start_main()+0xf5) [0x7f5da5a4c555] 11: (()+0x4c4383) [0x5566fef8a383] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -75> 2021-02-02 12:53:18.077371 7f5da959dec0 5 asok(0x55670a7881c0) register_command perfcounters_dump hook 0x55670a74e180 -74> 2021-02-02 12:53:18.077391 7f5da959dec0 5 asok(0x55670a7881c0) register_command 1 hook 0x55670a74e180 -73> 2021-02-02 12:53:18.077394 7f5da959dec0 5 asok(0x55670a7881c0) register_command perf dump hook 0x55670a74e180 -72> 2021-02-02 12:53:18.077398 7f5da959dec0 5 asok(0x55670a7881c0) register_command perfcounters_schema hook 0x55670a74e180 -71> 2021-02-02 12:53:18.077400 7f5da959dec0 5 asok(0x55670a7881c0) register_command perf histogram dump hook 0x55670a74e180 -70> 2021-02-02 12:53:18.077402 7f5da959dec0 5 asok(0x55670a7881c0) register_command 2 hook 0x55670a74e180 -69> 2021-02-02 12:53:18.077404 7f5da959dec0 5 asok(0x55670a7881c0) register_command perf schema hook 0x55670a74e180 -68> 2021-02-02 12:53:18.077407 7f5da959dec0 5 asok(0x55670a7881c0) register_command perf histogram schema hook 0x55670a74e180 -67> 2021-02-02 12:53:18.077410 7f5da959dec0 5 asok(0x55670a7881c0) register_command perf reset hook 0x55670a74e180 -66> 2021-02-02 12:53:18.077417 7f5da959dec0 5 asok(0x55670a7881c0) register_command config show hook 0x55670a74e180 -65> 2021-02-02 12:53:18.077453 7f5da959dec0 5 asok(0x55670a7881c0) register_command config help hook 0x55670a74e180 -64> 2021-02-02 12:53:18.077458 7f5da959dec0 5 asok(0x55670a7881c0) register_command config set hook 0x55670a74e180 -63> 2021-02-02 12:53:18.077465 7f5da959dec0 5 asok(0x55670a7881c0) register_command config get hook 0x55670a74e180 -62> 2021-02-02 12:53:18.077468 7f5da959dec0 5 asok(0x55670a7881c0) register_command config diff hook 0x55670a74e180 -61> 2021-02-02 12:53:18.077474 7f5da959dec0 5 asok(0x55670a7881c0) register_command config diff get hook 0x55670a74e180 -60> 2021-02-02 12:53:18.077482 7f5da959dec0 5 asok(0x55670a7881c0) register_command log flush hook 0x55670a74e180 -59> 2021-02-02 12:53:18.077485 7f5da959dec0 5 asok(0x55670a7881c0) register_command log dump hook 0x55670a74e180 -58> 2021-02-02 12:53:18.077489 7f5da959dec0 5 asok(0x55670a7881c0) register_command log reopen hook 0x55670a74e180 -57> 2021-02-02 12:53:18.077524 7f5da959dec0 5 asok(0x55670a7881c0) register_command dump_mempools hook 0x55670a9e4158 -56> 2021-02-02 12:53:18.084995 7f5da959dec0 0 set uid:gid to 167:167 (ceph:ceph) -55> 2021-02-02 12:53:18.085017 7f5da959dec0 0 ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous (stable), process ceph-osd, pid 374299 -54> 2021-02-02 12:53:18.085066 7f5da959dec0 5 object store type is bluestore -53> 2021-02-02 12:53:18.092759 7f5da3955700 2 Event(0x55670a9e8880 nevent=5000 time_id=1).set_owner idx=1 owner=140040153159424 -52> 2021-02-02 12:53:18.092786 7f5da3154700 2 Event(0x55670a9e8c80 nevent=5000 time_id=1).set_owner idx=2 owner=140040144766720 -51> 2021-02-02 12:53:18.092748 7f5da4156700 2 Event(0x55670a9e8480 nevent=5000 time_id=1).set_owner idx=0 owner=140040161552128 -50> 2021-02-02 12:53:18.093282 7f5da959dec0 1 -- 10.173.20.63:0/0 learned_addr learned my addr 10.173.20.63:0/0 -49> 2021-02-02 12:53:18.093298 7f5da959dec0 1 -- 10.173.20.63:6800/374299 _finish_bind bind my_inst.addr is 10.173.20.63:6800/374299 -48> 2021-02-02 12:53:18.093443 7f5da959dec0 1 -- 10.173.21.63:0/0 learned_addr learned my addr 10.173.21.63:0/0 -47> 2021-02-02 12:53:18.093457 7f5da959dec0 1 -- 10.173.21.63:6800/374299 _finish_bind bind my_inst.addr is 10.173.21.63:6800/374299 -46> 2021-02-02 12:53:18.093696 7f5da959dec0 1 -- 10.173.21.63:0/0 learned_addr learned my addr 10.173.21.63:0/0 -45> 2021-02-02 12:53:18.093714 7f5da959dec0 1 -- 10.173.21.63:6801/374299 _finish_bind bind my_inst.addr is 10.173.21.63:6801/374299 -44> 2021-02-02 12:53:18.094041 7f5da959dec0 1 -- 10.173.20.63:0/0 learned_addr learned my addr 10.173.20.63:0/0 -43> 2021-02-02 12:53:18.094056 7f5da959dec0 1 -- 10.173.20.63:6801/374299 _finish_bind bind my_inst.addr is 10.173.20.63:6801/374299 -42> 2021-02-02 12:53:18.094069 7f5da959dec0 0 pidfile_write: ignore empty --pid-file -41> 2021-02-02 12:53:18.131080 7f5da959dec0 5 asok(0x55670a7881c0) init /var/run/ceph/ceph-osd.108.asok -40> 2021-02-02 12:53:18.131092 7f5da959dec0 5 asok(0x55670a7881c0) bind_and_listen /var/run/ceph/ceph-osd.108.asok -39> 2021-02-02 12:53:18.131169 7f5da959dec0 5 asok(0x55670a7881c0) register_command 0 hook 0x55670a74c1d8 -38> 2021-02-02 12:53:18.131175 7f5da959dec0 5 asok(0x55670a7881c0) register_command version hook 0x55670a74c1d8 -37> 2021-02-02 12:53:18.131178 7f5da959dec0 5 asok(0x55670a7881c0) register_command git_version hook 0x55670a74c1d8 -36> 2021-02-02 12:53:18.131181 7f5da959dec0 5 asok(0x55670a7881c0) register_command help hook 0x55670a74e580 -35> 2021-02-02 12:53:18.131183 7f5da959dec0 5 asok(0x55670a7881c0) register_command get_command_descriptions hook 0x55670a74e590 -34> 2021-02-02 12:53:18.131253 7f5da959dec0 10 monclient: build_initial_monmap -33> 2021-02-02 12:53:18.131307 7f5da19d1700 5 asok(0x55670a7881c0) entry start -32> 2021-02-02 12:53:18.138962 7f5da959dec0 0 load: jerasure load: lrc load: isa -31> 2021-02-02 12:53:18.138991 7f5da959dec0 5 adding auth protocol: cephx -30> 2021-02-02 12:53:18.138996 7f5da959dec0 5 adding auth protocol: cephx -29> 2021-02-02 12:53:18.139055 7f5da959dec0 1 bdev create path /var/lib/ceph/osd/ceph-108/block type kernel -28> 2021-02-02 12:53:18.139067 7f5da959dec0 1 bdev(0x55670aad9200 /var/lib/ceph/osd/ceph-108/block) open path /var/lib/ceph/osd/ceph-108/block -27> 2021-02-02 12:53:18.139306 7f5da959dec0 1 bdev(0x55670aad9200 /var/lib/ceph/osd/ceph-108/block) open size 1000203091968 (0xe8e0c00000, 932GiB) block_size 4096 (4KiB) non-rotational -26> 2021-02-02 12:53:18.139515 7f5da959dec0 1 bluestore(/var/lib/ceph/osd/ceph-108) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2 -25> 2021-02-02 12:53:18.139546 7f5da959dec0 1 bdev(0x55670aad9200 /var/lib/ceph/osd/ceph-108/block) close -24> 2021-02-02 12:53:18.399086 7f5da959dec0 5 asok(0x55670a7881c0) register_command objecter_requests hook 0x55670a74e610 -23> 2021-02-02 12:53:18.399142 7f5da959dec0 1 -- 10.173.20.63:6800/374299 start start -22> 2021-02-02 12:53:18.399149 7f5da959dec0 1 -- - start start -21> 2021-02-02 12:53:18.399152 7f5da959dec0 1 -- - start start -20> 2021-02-02 12:53:18.399171 7f5da959dec0 1 -- 10.173.20.63:6801/374299 start start -19> 2021-02-02 12:53:18.399175 7f5da959dec0 1 -- 10.173.21.63:6801/374299 start start -18> 2021-02-02 12:53:18.399186 7f5da959dec0 1 -- 10.173.21.63:6800/374299 start start -17> 2021-02-02 12:53:18.399189 7f5da959dec0 1 -- - start start -16> 2021-02-02 12:53:18.399613 7f5da959dec0 2 osd.108 0 init /var/lib/ceph/osd/ceph-108 (looks like ssd) -15> 2021-02-02 12:53:18.399631 7f5da959dec0 2 osd.108 0 journal /var/lib/ceph/osd/ceph-108/journal -14> 2021-02-02 12:53:18.399641 7f5da959dec0 1 bluestore(/var/lib/ceph/osd/ceph-108) _mount path /var/lib/ceph/osd/ceph-108 -13> 2021-02-02 12:53:18.399991 7f5da959dec0 1 bdev create path /var/lib/ceph/osd/ceph-108/block type kernel -12> 2021-02-02 12:53:18.400002 7f5da959dec0 1 bdev(0x55670aad8600 /var/lib/ceph/osd/ceph-108/block) open path /var/lib/ceph/osd/ceph-108/block -11> 2021-02-02 12:53:18.400156 7f5da959dec0 1 bdev(0x55670aad8600 /var/lib/ceph/osd/ceph-108/block) open size 1000203091968 (0xe8e0c00000, 932GiB) block_size 4096 (4KiB) non-rotational -10> 2021-02-02 12:53:18.400413 7f5da959dec0 1 bluestore(/var/lib/ceph/osd/ceph-108) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2 -9> 2021-02-02 12:53:18.400477 7f5da959dec0 5 asok(0x55670a7881c0) register_command bluestore bluefs available hook 0x55670a74e620 -8> 2021-02-02 12:53:18.400564 7f5da959dec0 1 bdev create path /var/lib/ceph/osd/ceph-108/block type kernel -7> 2021-02-02 12:53:18.400571 7f5da959dec0 1 bdev(0x55670aad6600 /var/lib/ceph/osd/ceph-108/block) open path /var/lib/ceph/osd/ceph-108/block -6> 2021-02-02 12:53:18.400716 7f5da959dec0 1 bdev(0x55670aad6600 /var/lib/ceph/osd/ceph-108/block) open size 1000203091968 (0xe8e0c00000, 932GiB) block_size 4096 (4KiB) non-rotational -5> 2021-02-02 12:53:18.400731 7f5da959dec0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-108/block size 932GiB -4> 2021-02-02 12:53:18.400758 7f5da959dec0 1 bluefs mount -3> 2021-02-02 12:53:18.401032 7f5da959dec0 1 bluefs _init_alloc id 1 alloc_size 0x10000 size 0xe8e0c00000 -2> 2021-02-02 12:53:18.401048 7f5da959dec0 5 asok(0x55670a7881c0) register_command bluestore allocator dump bluefs-db hook 0x55670aad3660 -1> 2021-02-02 12:53:18.401057 7f5da959dec0 5 asok(0x55670a7881c0) register_command bluestore allocator score bluefs-db hook 0x55670aad3660 0> 2021-02-02 13:21:09.304110 7f5da959dec0 -1 *** Caught signal (Segmentation fault) ** in thread 7f5da959dec0 thread_name:ceph-osd ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous (stable) 1: (()+0xa76e51) [0x5566ff53ce51] 2: (()+0xf630) [0x7f5da6a40630] 3: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x452) [0x5566ff4efcd2] 4: (BlueFS::_replay(bool)+0x48d) [0x5566ff50378d] 5: (BlueFS::mount()+0x1d4) [0x5566ff507374] 6: (BlueStore::_open_db(bool)+0x1857) [0x5566ff415827] 7: (BlueStore::_mount(bool)+0x2f6) [0x5566ff44acf6] 8: (OSD::init()+0x472) [0x5566fefe9762] 9: (main()+0x2f49) [0x5566feeea719] 10: (__libc_start_main()+0xf5) [0x7f5da5a4c555] 11: (()+0x4c4383) [0x5566fef8a383] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.108.log --- end dump of recent events ---