2019-03-14 07:46:55.989364 7fd5fd36dd00 0 set uid:gid to 167:167 (ceph:ceph) 2019-03-14 07:46:55.989376 7fd5fd36dd00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124558 2019-03-14 07:46:55.992090 7fd5fd36dd00 1 bluestore(/var/lib/ceph/tmp/mnt.BUaW4v) mkfs path /var/lib/ceph/tmp/mnt.BUaW4v 2019-03-14 07:46:55.993050 7fd5fd36dd00 1 bdev create path /var/lib/ceph/tmp/mnt.BUaW4v/block type kernel 2019-03-14 07:46:55.993064 7fd5fd36dd00 1 bdev(0x5654beb11c00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open path /var/lib/ceph/tmp/mnt.BUaW4v/block 2019-03-14 07:46:55.993337 7fd5fd36dd00 1 bdev(0x5654beb11c00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:56.000939 7fd5fd36dd00 1 bluestore(/var/lib/ceph/tmp/mnt.BUaW4v) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-03-14 07:46:56.001023 7fd5fd36dd00 1 bdev create path /var/lib/ceph/tmp/mnt.BUaW4v/block type kernel 2019-03-14 07:46:56.001030 7fd5fd36dd00 1 bdev(0x5654beb11e00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open path /var/lib/ceph/tmp/mnt.BUaW4v/block 2019-03-14 07:46:56.001427 7fd5fd36dd00 1 bdev(0x5654beb11e00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:56.001434 7fd5fd36dd00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.BUaW4v/block size 3.64TiB 2019-03-14 07:46:56.001438 7fd5fd36dd00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 2019-03-14 07:46:56.001470 7fd5fd36dd00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 2019-03-14 07:46:56.001501 7fd5fd36dd00 1 bluefs mkfs uuid 17611111-26e1-466d-b6a1-caf1b1af06d9 2019-03-14 07:46:56.034238 7fd5fd36dd00 1 stupidalloc 0x0x5654beb06530 shutdown 2019-03-14 07:46:56.034259 7fd5fd36dd00 1 bluefs mount 2019-03-14 07:46:56.059112 7fd5fd36dd00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7fd5fd36dd00 time 2019-03-14 07:46:56.055966 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x5654b3e18aae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5654b3d92224] 3: (BlueFS::_replay(bool)+0x2e9) [0x5654b3da4949] 4: (BlueFS::mount()+0x1cf) [0x5654b3da867f] 5: (BlueStore::_open_db(bool)+0x1746) [0x5654b3cc31d6] 6: (BlueStore::mkfs()+0xf5d) [0x5654b3ccb57d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5654b38a9d55] 8: (main()+0x114e) [0x5654b37bf86e] 9: (__libc_start_main()+0xf5) [0x7fd5fa62b725] 10: (_start()+0x29) [0x5654b3861389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -59> 2019-03-14 07:46:55.982143 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perfcounters_dump hook 0x5654beac8190 -58> 2019-03-14 07:46:55.982156 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command 1 hook 0x5654beac8190 -57> 2019-03-14 07:46:55.982158 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perf dump hook 0x5654beac8190 -56> 2019-03-14 07:46:55.982166 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perfcounters_schema hook 0x5654beac8190 -55> 2019-03-14 07:46:55.982168 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perf histogram dump hook 0x5654beac8190 -54> 2019-03-14 07:46:55.982170 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command 2 hook 0x5654beac8190 -53> 2019-03-14 07:46:55.982171 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perf schema hook 0x5654beac8190 -52> 2019-03-14 07:46:55.982172 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perf histogram schema hook 0x5654beac8190 -51> 2019-03-14 07:46:55.982174 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command perf reset hook 0x5654beac8190 -50> 2019-03-14 07:46:55.982176 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command config show hook 0x5654beac8190 -49> 2019-03-14 07:46:55.982178 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command config help hook 0x5654beac8190 -48> 2019-03-14 07:46:55.982180 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command config set hook 0x5654beac8190 -47> 2019-03-14 07:46:55.982182 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command config get hook 0x5654beac8190 -46> 2019-03-14 07:46:55.982185 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command config diff hook 0x5654beac8190 -45> 2019-03-14 07:46:55.982187 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command config diff get hook 0x5654beac8190 -44> 2019-03-14 07:46:55.982190 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command log flush hook 0x5654beac8190 -43> 2019-03-14 07:46:55.982194 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command log dump hook 0x5654beac8190 -42> 2019-03-14 07:46:55.982197 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command log reopen hook 0x5654beac8190 -41> 2019-03-14 07:46:55.982207 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command dump_mempools hook 0x5654bece74a8 -40> 2019-03-14 07:46:55.989364 7fd5fd36dd00 0 set uid:gid to 167:167 (ceph:ceph) -39> 2019-03-14 07:46:55.989376 7fd5fd36dd00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124558 -38> 2019-03-14 07:46:55.989413 7fd5fd36dd00 5 object store type is bluestore -37> 2019-03-14 07:46:55.991055 7fd5fd36dd00 5 asok(0x5654beb281c0) init /var/run/ceph/ceph-osd.45.asok -36> 2019-03-14 07:46:55.991069 7fd5fd36dd00 5 asok(0x5654beb281c0) bind_and_listen /var/run/ceph/ceph-osd.45.asok -35> 2019-03-14 07:46:55.991091 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command 0 hook 0x5654beac6180 -34> 2019-03-14 07:46:55.991095 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command version hook 0x5654beac6180 -33> 2019-03-14 07:46:55.991098 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command git_version hook 0x5654beac6180 -32> 2019-03-14 07:46:55.991101 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command help hook 0x5654beac8260 -31> 2019-03-14 07:46:55.991105 7fd5fd36dd00 5 asok(0x5654beb281c0) register_command get_command_descriptions hook 0x5654beac8270 -30> 2019-03-14 07:46:55.991159 7fd5fd36dd00 10 monclient: build_initial_monmap -29> 2019-03-14 07:46:55.991180 7fd5f7b90700 5 asok(0x5654beb281c0) entry start -28> 2019-03-14 07:46:55.991202 7fd5fd36dd00 10 monclient: get_monmap_privately -27> 2019-03-14 07:46:55.991766 7fd5f738f700 2 Event(0x5654beb11080 nevent=5000 time_id=1).set_owner idx=0 owner=140556952467200 -26> 2019-03-14 07:46:55.991778 7fd5f638d700 2 Event(0x5654beb11880 nevent=5000 time_id=1).set_owner idx=2 owner=140556935681792 -25> 2019-03-14 07:46:55.991783 7fd5f6b8e700 2 Event(0x5654beb11480 nevent=5000 time_id=1).set_owner idx=1 owner=140556944074496 -24> 2019-03-14 07:46:55.991853 7fd5fd36dd00 1 Processor -- start -23> 2019-03-14 07:46:55.991921 7fd5fd36dd00 1 -- - start start -22> 2019-03-14 07:46:55.991927 7fd5fd36dd00 10 monclient: have 1 fsid 04d7f947-8bbd-3880-9d65-252b11e4617c -21> 2019-03-14 07:46:55.991951 7fd5fd36dd00 1 -- - shutdown_connections -20> 2019-03-14 07:46:55.992024 7fd5fd36dd00 1 -- - shutdown_connections -19> 2019-03-14 07:46:55.992046 7fd5fd36dd00 1 -- - wait complete. -18> 2019-03-14 07:46:55.992050 7fd5fd36dd00 1 -- - >> - conn(0x5654bee86000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -17> 2019-03-14 07:46:55.992061 7fd5fd36dd00 2 -- - >> - conn(0x5654bee86000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -16> 2019-03-14 07:46:55.992090 7fd5fd36dd00 1 bluestore(/var/lib/ceph/tmp/mnt.BUaW4v) mkfs path /var/lib/ceph/tmp/mnt.BUaW4v -15> 2019-03-14 07:46:55.992658 7fd5fd36dd00 2 bluestore(/var/lib/ceph/tmp/mnt.BUaW4v/block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding -14> 2019-03-14 07:46:55.992899 7fd5fd36dd00 2 bluestore(/var/lib/ceph/tmp/mnt.BUaW4v/block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding -13> 2019-03-14 07:46:55.993050 7fd5fd36dd00 1 bdev create path /var/lib/ceph/tmp/mnt.BUaW4v/block type kernel -12> 2019-03-14 07:46:55.993064 7fd5fd36dd00 1 bdev(0x5654beb11c00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open path /var/lib/ceph/tmp/mnt.BUaW4v/block -11> 2019-03-14 07:46:55.993337 7fd5fd36dd00 1 bdev(0x5654beb11c00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -10> 2019-03-14 07:46:56.000939 7fd5fd36dd00 1 bluestore(/var/lib/ceph/tmp/mnt.BUaW4v) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -9> 2019-03-14 07:46:56.001023 7fd5fd36dd00 1 bdev create path /var/lib/ceph/tmp/mnt.BUaW4v/block type kernel -8> 2019-03-14 07:46:56.001030 7fd5fd36dd00 1 bdev(0x5654beb11e00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open path /var/lib/ceph/tmp/mnt.BUaW4v/block -7> 2019-03-14 07:46:56.001427 7fd5fd36dd00 1 bdev(0x5654beb11e00 /var/lib/ceph/tmp/mnt.BUaW4v/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -6> 2019-03-14 07:46:56.001434 7fd5fd36dd00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.BUaW4v/block size 3.64TiB -5> 2019-03-14 07:46:56.001438 7fd5fd36dd00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 -4> 2019-03-14 07:46:56.001470 7fd5fd36dd00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 -3> 2019-03-14 07:46:56.001501 7fd5fd36dd00 1 bluefs mkfs uuid 17611111-26e1-466d-b6a1-caf1b1af06d9 -2> 2019-03-14 07:46:56.034238 7fd5fd36dd00 1 stupidalloc 0x0x5654beb06530 shutdown -1> 2019-03-14 07:46:56.034259 7fd5fd36dd00 1 bluefs mount 0> 2019-03-14 07:46:56.059112 7fd5fd36dd00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7fd5fd36dd00 time 2019-03-14 07:46:56.055966 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x5654b3e18aae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5654b3d92224] 3: (BlueFS::_replay(bool)+0x2e9) [0x5654b3da4949] 4: (BlueFS::mount()+0x1cf) [0x5654b3da867f] 5: (BlueStore::_open_db(bool)+0x1746) [0x5654b3cc31d6] 6: (BlueStore::mkfs()+0xf5d) [0x5654b3ccb57d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5654b38a9d55] 8: (main()+0x114e) [0x5654b37bf86e] 9: (__libc_start_main()+0xf5) [0x7fd5fa62b725] 10: (_start()+0x29) [0x5654b3861389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:56.062863 7fd5fd36dd00 -1 *** Caught signal (Aborted) ** in thread 7fd5fd36dd00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x5654b3ddbd29] 2: (()+0x10c10) [0x7fd5fb679c10] 3: (gsignal()+0x37) [0x7fd5fa63ff67] 4: (abort()+0x13a) [0x7fd5fa64133a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x5654b3e18c20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5654b3d92224] 7: (BlueFS::_replay(bool)+0x2e9) [0x5654b3da4949] 8: (BlueFS::mount()+0x1cf) [0x5654b3da867f] 9: (BlueStore::_open_db(bool)+0x1746) [0x5654b3cc31d6] 10: (BlueStore::mkfs()+0xf5d) [0x5654b3ccb57d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5654b38a9d55] 12: (main()+0x114e) [0x5654b37bf86e] 13: (__libc_start_main()+0xf5) [0x7fd5fa62b725] 14: (_start()+0x29) [0x5654b3861389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2019-03-14 07:46:56.062863 7fd5fd36dd00 -1 *** Caught signal (Aborted) ** in thread 7fd5fd36dd00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x5654b3ddbd29] 2: (()+0x10c10) [0x7fd5fb679c10] 3: (gsignal()+0x37) [0x7fd5fa63ff67] 4: (abort()+0x13a) [0x7fd5fa64133a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x5654b3e18c20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5654b3d92224] 7: (BlueFS::_replay(bool)+0x2e9) [0x5654b3da4949] 8: (BlueFS::mount()+0x1cf) [0x5654b3da867f] 9: (BlueStore::_open_db(bool)+0x1746) [0x5654b3cc31d6] 10: (BlueStore::mkfs()+0xf5d) [0x5654b3ccb57d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5654b38a9d55] 12: (main()+0x114e) [0x5654b37bf86e] 13: (__libc_start_main()+0xf5) [0x7fd5fa62b725] 14: (_start()+0x29) [0x5654b3861389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:57.092163 7f24610b6d00 0 set uid:gid to 167:167 (ceph:ceph) 2019-03-14 07:46:57.092183 7f24610b6d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124698 2019-03-14 07:46:57.095078 7f24610b6d00 1 bluestore(/var/lib/ceph/tmp/mnt.EGdqgI) mkfs path /var/lib/ceph/tmp/mnt.EGdqgI 2019-03-14 07:46:57.096144 7f24610b6d00 1 bdev create path /var/lib/ceph/tmp/mnt.EGdqgI/block type kernel 2019-03-14 07:46:57.096157 7f24610b6d00 1 bdev(0x5566cca2bc00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open path /var/lib/ceph/tmp/mnt.EGdqgI/block 2019-03-14 07:46:57.096460 7f24610b6d00 1 bdev(0x5566cca2bc00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:57.100721 7f24610b6d00 1 bluestore(/var/lib/ceph/tmp/mnt.EGdqgI) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-03-14 07:46:57.100762 7f24610b6d00 1 bdev create path /var/lib/ceph/tmp/mnt.EGdqgI/block type kernel 2019-03-14 07:46:57.100765 7f24610b6d00 1 bdev(0x5566cca2be00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open path /var/lib/ceph/tmp/mnt.EGdqgI/block 2019-03-14 07:46:57.101690 7f24610b6d00 1 bdev(0x5566cca2be00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:57.101700 7f24610b6d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.EGdqgI/block size 3.64TiB 2019-03-14 07:46:57.101705 7f24610b6d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 2019-03-14 07:46:57.101727 7f24610b6d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 2019-03-14 07:46:57.101761 7f24610b6d00 1 bluefs mkfs uuid 6fb73576-a2a2-407f-b461-c8ffbfe5f304 2019-03-14 07:46:57.134045 7f24610b6d00 1 stupidalloc 0x0x5566cca205a0 shutdown 2019-03-14 07:46:57.134069 7f24610b6d00 1 bluefs mount 2019-03-14 07:46:57.167038 7f24610b6d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f24610b6d00 time 2019-03-14 07:46:57.164144 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x5566c19d3aae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5566c194d224] 3: (BlueFS::_replay(bool)+0x2e9) [0x5566c195f949] 4: (BlueFS::mount()+0x1cf) [0x5566c196367f] 5: (BlueStore::_open_db(bool)+0x1746) [0x5566c187e1d6] 6: (BlueStore::mkfs()+0xf5d) [0x5566c188657d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5566c1464d55] 8: (main()+0x114e) [0x5566c137a86e] 9: (__libc_start_main()+0xf5) [0x7f245e374725] 10: (_start()+0x29) [0x5566c141c389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -57> 2019-03-14 07:46:57.084658 7f24610b6d00 5 asok(0x5566cca421c0) register_command perfcounters_dump hook 0x5566cc9e2190 -56> 2019-03-14 07:46:57.084670 7f24610b6d00 5 asok(0x5566cca421c0) register_command 1 hook 0x5566cc9e2190 -55> 2019-03-14 07:46:57.084672 7f24610b6d00 5 asok(0x5566cca421c0) register_command perf dump hook 0x5566cc9e2190 -54> 2019-03-14 07:46:57.084680 7f24610b6d00 5 asok(0x5566cca421c0) register_command perfcounters_schema hook 0x5566cc9e2190 -53> 2019-03-14 07:46:57.084685 7f24610b6d00 5 asok(0x5566cca421c0) register_command perf histogram dump hook 0x5566cc9e2190 -52> 2019-03-14 07:46:57.084686 7f24610b6d00 5 asok(0x5566cca421c0) register_command 2 hook 0x5566cc9e2190 -51> 2019-03-14 07:46:57.084689 7f24610b6d00 5 asok(0x5566cca421c0) register_command perf schema hook 0x5566cc9e2190 -50> 2019-03-14 07:46:57.084692 7f24610b6d00 5 asok(0x5566cca421c0) register_command perf histogram schema hook 0x5566cc9e2190 -49> 2019-03-14 07:46:57.084699 7f24610b6d00 5 asok(0x5566cca421c0) register_command perf reset hook 0x5566cc9e2190 -48> 2019-03-14 07:46:57.084701 7f24610b6d00 5 asok(0x5566cca421c0) register_command config show hook 0x5566cc9e2190 -47> 2019-03-14 07:46:57.084704 7f24610b6d00 5 asok(0x5566cca421c0) register_command config help hook 0x5566cc9e2190 -46> 2019-03-14 07:46:57.084706 7f24610b6d00 5 asok(0x5566cca421c0) register_command config set hook 0x5566cc9e2190 -45> 2019-03-14 07:46:57.084709 7f24610b6d00 5 asok(0x5566cca421c0) register_command config get hook 0x5566cc9e2190 -44> 2019-03-14 07:46:57.084711 7f24610b6d00 5 asok(0x5566cca421c0) register_command config diff hook 0x5566cc9e2190 -43> 2019-03-14 07:46:57.084714 7f24610b6d00 5 asok(0x5566cca421c0) register_command config diff get hook 0x5566cc9e2190 -42> 2019-03-14 07:46:57.084716 7f24610b6d00 5 asok(0x5566cca421c0) register_command log flush hook 0x5566cc9e2190 -41> 2019-03-14 07:46:57.084719 7f24610b6d00 5 asok(0x5566cca421c0) register_command log dump hook 0x5566cc9e2190 -40> 2019-03-14 07:46:57.084721 7f24610b6d00 5 asok(0x5566cca421c0) register_command log reopen hook 0x5566cc9e2190 -39> 2019-03-14 07:46:57.084728 7f24610b6d00 5 asok(0x5566cca421c0) register_command dump_mempools hook 0x5566ccc014a8 -38> 2019-03-14 07:46:57.092163 7f24610b6d00 0 set uid:gid to 167:167 (ceph:ceph) -37> 2019-03-14 07:46:57.092183 7f24610b6d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124698 -36> 2019-03-14 07:46:57.092229 7f24610b6d00 5 object store type is bluestore -35> 2019-03-14 07:46:57.094094 7f24610b6d00 5 asok(0x5566cca421c0) init /var/run/ceph/ceph-osd.45.asok -34> 2019-03-14 07:46:57.094106 7f24610b6d00 5 asok(0x5566cca421c0) bind_and_listen /var/run/ceph/ceph-osd.45.asok -33> 2019-03-14 07:46:57.094164 7f24610b6d00 5 asok(0x5566cca421c0) register_command 0 hook 0x5566cc9e0180 -32> 2019-03-14 07:46:57.094170 7f24610b6d00 5 asok(0x5566cca421c0) register_command version hook 0x5566cc9e0180 -31> 2019-03-14 07:46:57.094174 7f24610b6d00 5 asok(0x5566cca421c0) register_command git_version hook 0x5566cc9e0180 -30> 2019-03-14 07:46:57.094179 7f24610b6d00 5 asok(0x5566cca421c0) register_command help hook 0x5566cc9e2260 -29> 2019-03-14 07:46:57.094183 7f24610b6d00 5 asok(0x5566cca421c0) register_command get_command_descriptions hook 0x5566cc9e2270 -28> 2019-03-14 07:46:57.094241 7f24610b6d00 10 monclient: build_initial_monmap -27> 2019-03-14 07:46:57.094251 7f245b8d9700 5 asok(0x5566cca421c0) entry start -26> 2019-03-14 07:46:57.094284 7f24610b6d00 10 monclient: get_monmap_privately -25> 2019-03-14 07:46:57.094761 7f245a8d7700 2 Event(0x5566cca2b480 nevent=5000 time_id=1).set_owner idx=1 owner=139794114770688 -24> 2019-03-14 07:46:57.094769 7f245a0d6700 2 Event(0x5566cca2b880 nevent=5000 time_id=1).set_owner idx=2 owner=139794106377984 -23> 2019-03-14 07:46:57.094756 7f245b0d8700 2 Event(0x5566cca2b080 nevent=5000 time_id=1).set_owner idx=0 owner=139794123163392 -22> 2019-03-14 07:46:57.094828 7f24610b6d00 1 Processor -- start -21> 2019-03-14 07:46:57.094899 7f24610b6d00 1 -- - start start -20> 2019-03-14 07:46:57.094902 7f24610b6d00 10 monclient: have 1 fsid 04d7f947-8bbd-3880-9d65-252b11e4617c -19> 2019-03-14 07:46:57.094928 7f24610b6d00 1 -- - shutdown_connections -18> 2019-03-14 07:46:57.095003 7f24610b6d00 1 -- - shutdown_connections -17> 2019-03-14 07:46:57.095026 7f24610b6d00 1 -- - wait complete. -16> 2019-03-14 07:46:57.095030 7f24610b6d00 1 -- - >> - conn(0x5566ccd9c000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -15> 2019-03-14 07:46:57.095041 7f24610b6d00 2 -- - >> - conn(0x5566ccd9c000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -14> 2019-03-14 07:46:57.095078 7f24610b6d00 1 bluestore(/var/lib/ceph/tmp/mnt.EGdqgI) mkfs path /var/lib/ceph/tmp/mnt.EGdqgI -13> 2019-03-14 07:46:57.096144 7f24610b6d00 1 bdev create path /var/lib/ceph/tmp/mnt.EGdqgI/block type kernel -12> 2019-03-14 07:46:57.096157 7f24610b6d00 1 bdev(0x5566cca2bc00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open path /var/lib/ceph/tmp/mnt.EGdqgI/block -11> 2019-03-14 07:46:57.096460 7f24610b6d00 1 bdev(0x5566cca2bc00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -10> 2019-03-14 07:46:57.100721 7f24610b6d00 1 bluestore(/var/lib/ceph/tmp/mnt.EGdqgI) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -9> 2019-03-14 07:46:57.100762 7f24610b6d00 1 bdev create path /var/lib/ceph/tmp/mnt.EGdqgI/block type kernel -8> 2019-03-14 07:46:57.100765 7f24610b6d00 1 bdev(0x5566cca2be00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open path /var/lib/ceph/tmp/mnt.EGdqgI/block -7> 2019-03-14 07:46:57.101690 7f24610b6d00 1 bdev(0x5566cca2be00 /var/lib/ceph/tmp/mnt.EGdqgI/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -6> 2019-03-14 07:46:57.101700 7f24610b6d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.EGdqgI/block size 3.64TiB -5> 2019-03-14 07:46:57.101705 7f24610b6d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 -4> 2019-03-14 07:46:57.101727 7f24610b6d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 -3> 2019-03-14 07:46:57.101761 7f24610b6d00 1 bluefs mkfs uuid 6fb73576-a2a2-407f-b461-c8ffbfe5f304 -2> 2019-03-14 07:46:57.134045 7f24610b6d00 1 stupidalloc 0x0x5566cca205a0 shutdown -1> 2019-03-14 07:46:57.134069 7f24610b6d00 1 bluefs mount 0> 2019-03-14 07:46:57.167038 7f24610b6d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f24610b6d00 time 2019-03-14 07:46:57.164144 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x5566c19d3aae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5566c194d224] 3: (BlueFS::_replay(bool)+0x2e9) [0x5566c195f949] 4: (BlueFS::mount()+0x1cf) [0x5566c196367f] 5: (BlueStore::_open_db(bool)+0x1746) [0x5566c187e1d6] 6: (BlueStore::mkfs()+0xf5d) [0x5566c188657d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5566c1464d55] 8: (main()+0x114e) [0x5566c137a86e] 9: (__libc_start_main()+0xf5) [0x7f245e374725] 10: (_start()+0x29) [0x5566c141c389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:57.170587 7f24610b6d00 -1 *** Caught signal (Aborted) ** in thread 7f24610b6d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x5566c1996d29] 2: (()+0x10c10) [0x7f245f3c2c10] 3: (gsignal()+0x37) [0x7f245e388f67] 4: (abort()+0x13a) [0x7f245e38a33a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x5566c19d3c20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5566c194d224] 7: (BlueFS::_replay(bool)+0x2e9) [0x5566c195f949] 8: (BlueFS::mount()+0x1cf) [0x5566c196367f] 9: (BlueStore::_open_db(bool)+0x1746) [0x5566c187e1d6] 10: (BlueStore::mkfs()+0xf5d) [0x5566c188657d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5566c1464d55] 12: (main()+0x114e) [0x5566c137a86e] 13: (__libc_start_main()+0xf5) [0x7f245e374725] 14: (_start()+0x29) [0x5566c141c389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2019-03-14 07:46:57.170587 7f24610b6d00 -1 *** Caught signal (Aborted) ** in thread 7f24610b6d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x5566c1996d29] 2: (()+0x10c10) [0x7f245f3c2c10] 3: (gsignal()+0x37) [0x7f245e388f67] 4: (abort()+0x13a) [0x7f245e38a33a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x5566c19d3c20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x5566c194d224] 7: (BlueFS::_replay(bool)+0x2e9) [0x5566c195f949] 8: (BlueFS::mount()+0x1cf) [0x5566c196367f] 9: (BlueStore::_open_db(bool)+0x1746) [0x5566c187e1d6] 10: (BlueStore::mkfs()+0xf5d) [0x5566c188657d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x5566c1464d55] 12: (main()+0x114e) [0x5566c137a86e] 13: (__libc_start_main()+0xf5) [0x7f245e374725] 14: (_start()+0x29) [0x5566c141c389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:57.862948 7f32d2496d00 0 set uid:gid to 167:167 (ceph:ceph) 2019-03-14 07:46:57.862969 7f32d2496d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124758 2019-03-14 07:46:57.866097 7f32d2496d00 1 bluestore(/var/lib/ceph/tmp/mnt.cb9gUS) mkfs path /var/lib/ceph/tmp/mnt.cb9gUS 2019-03-14 07:46:57.867132 7f32d2496d00 1 bdev create path /var/lib/ceph/tmp/mnt.cb9gUS/block type kernel 2019-03-14 07:46:57.867148 7f32d2496d00 1 bdev(0x55e194d87c00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open path /var/lib/ceph/tmp/mnt.cb9gUS/block 2019-03-14 07:46:57.867485 7f32d2496d00 1 bdev(0x55e194d87c00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:57.875626 7f32d2496d00 1 bluestore(/var/lib/ceph/tmp/mnt.cb9gUS) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-03-14 07:46:57.875668 7f32d2496d00 1 bdev create path /var/lib/ceph/tmp/mnt.cb9gUS/block type kernel 2019-03-14 07:46:57.875670 7f32d2496d00 1 bdev(0x55e194d87e00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open path /var/lib/ceph/tmp/mnt.cb9gUS/block 2019-03-14 07:46:57.876137 7f32d2496d00 1 bdev(0x55e194d87e00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:57.876147 7f32d2496d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.cb9gUS/block size 3.64TiB 2019-03-14 07:46:57.876155 7f32d2496d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 2019-03-14 07:46:57.876173 7f32d2496d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 2019-03-14 07:46:57.876205 7f32d2496d00 1 bluefs mkfs uuid c654e950-3e1e-4629-9489-acbbb7d16212 2019-03-14 07:46:57.908926 7f32d2496d00 1 stupidalloc 0x0x55e194d7c530 shutdown 2019-03-14 07:46:57.908960 7f32d2496d00 1 bluefs mount 2019-03-14 07:46:57.934114 7f32d2496d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f32d2496d00 time 2019-03-14 07:46:57.930656 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x55e18b70baae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x55e18b685224] 3: (BlueFS::_replay(bool)+0x2e9) [0x55e18b697949] 4: (BlueFS::mount()+0x1cf) [0x55e18b69b67f] 5: (BlueStore::_open_db(bool)+0x1746) [0x55e18b5b61d6] 6: (BlueStore::mkfs()+0xf5d) [0x55e18b5be57d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x55e18b19cd55] 8: (main()+0x114e) [0x55e18b0b286e] 9: (__libc_start_main()+0xf5) [0x7f32cf754725] 10: (_start()+0x29) [0x55e18b154389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -57> 2019-03-14 07:46:57.855887 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perfcounters_dump hook 0x55e194d3e190 -56> 2019-03-14 07:46:57.855899 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command 1 hook 0x55e194d3e190 -55> 2019-03-14 07:46:57.855901 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perf dump hook 0x55e194d3e190 -54> 2019-03-14 07:46:57.855908 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perfcounters_schema hook 0x55e194d3e190 -53> 2019-03-14 07:46:57.855910 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perf histogram dump hook 0x55e194d3e190 -52> 2019-03-14 07:46:57.855911 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command 2 hook 0x55e194d3e190 -51> 2019-03-14 07:46:57.855913 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perf schema hook 0x55e194d3e190 -50> 2019-03-14 07:46:57.855915 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perf histogram schema hook 0x55e194d3e190 -49> 2019-03-14 07:46:57.855920 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command perf reset hook 0x55e194d3e190 -48> 2019-03-14 07:46:57.855923 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command config show hook 0x55e194d3e190 -47> 2019-03-14 07:46:57.855926 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command config help hook 0x55e194d3e190 -46> 2019-03-14 07:46:57.855929 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command config set hook 0x55e194d3e190 -45> 2019-03-14 07:46:57.855931 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command config get hook 0x55e194d3e190 -44> 2019-03-14 07:46:57.855933 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command config diff hook 0x55e194d3e190 -43> 2019-03-14 07:46:57.855935 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command config diff get hook 0x55e194d3e190 -42> 2019-03-14 07:46:57.855938 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command log flush hook 0x55e194d3e190 -41> 2019-03-14 07:46:57.855940 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command log dump hook 0x55e194d3e190 -40> 2019-03-14 07:46:57.855943 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command log reopen hook 0x55e194d3e190 -39> 2019-03-14 07:46:57.855949 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command dump_mempools hook 0x55e194f5d4a8 -38> 2019-03-14 07:46:57.862948 7f32d2496d00 0 set uid:gid to 167:167 (ceph:ceph) -37> 2019-03-14 07:46:57.862969 7f32d2496d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124758 -36> 2019-03-14 07:46:57.863028 7f32d2496d00 5 object store type is bluestore -35> 2019-03-14 07:46:57.865095 7f32d2496d00 5 asok(0x55e194d9e1c0) init /var/run/ceph/ceph-osd.45.asok -34> 2019-03-14 07:46:57.865113 7f32d2496d00 5 asok(0x55e194d9e1c0) bind_and_listen /var/run/ceph/ceph-osd.45.asok -33> 2019-03-14 07:46:57.865171 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command 0 hook 0x55e194d3c180 -32> 2019-03-14 07:46:57.865178 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command version hook 0x55e194d3c180 -31> 2019-03-14 07:46:57.865181 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command git_version hook 0x55e194d3c180 -30> 2019-03-14 07:46:57.865187 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command help hook 0x55e194d3e260 -29> 2019-03-14 07:46:57.865191 7f32d2496d00 5 asok(0x55e194d9e1c0) register_command get_command_descriptions hook 0x55e194d3e270 -28> 2019-03-14 07:46:57.865243 7f32d2496d00 10 monclient: build_initial_monmap -27> 2019-03-14 07:46:57.865275 7f32d2496d00 10 monclient: get_monmap_privately -26> 2019-03-14 07:46:57.865280 7f32cccb9700 5 asok(0x55e194d9e1c0) entry start -25> 2019-03-14 07:46:57.865760 7f32cc4b8700 2 Event(0x55e194d87080 nevent=5000 time_id=1).set_owner idx=0 owner=139856152594176 -24> 2019-03-14 07:46:57.865785 7f32cbcb7700 2 Event(0x55e194d87480 nevent=5000 time_id=1).set_owner idx=1 owner=139856144201472 -23> 2019-03-14 07:46:57.865806 7f32cb4b6700 2 Event(0x55e194d87880 nevent=5000 time_id=1).set_owner idx=2 owner=139856135808768 -22> 2019-03-14 07:46:57.865849 7f32d2496d00 1 Processor -- start -21> 2019-03-14 07:46:57.865907 7f32d2496d00 1 -- - start start -20> 2019-03-14 07:46:57.865931 7f32d2496d00 10 monclient: have 1 fsid 04d7f947-8bbd-3880-9d65-252b11e4617c -19> 2019-03-14 07:46:57.865944 7f32d2496d00 1 -- - shutdown_connections -18> 2019-03-14 07:46:57.866021 7f32d2496d00 1 -- - shutdown_connections -17> 2019-03-14 07:46:57.866043 7f32d2496d00 1 -- - wait complete. -16> 2019-03-14 07:46:57.866047 7f32d2496d00 1 -- - >> - conn(0x55e1950f8000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -15> 2019-03-14 07:46:57.866064 7f32d2496d00 2 -- - >> - conn(0x55e1950f8000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -14> 2019-03-14 07:46:57.866097 7f32d2496d00 1 bluestore(/var/lib/ceph/tmp/mnt.cb9gUS) mkfs path /var/lib/ceph/tmp/mnt.cb9gUS -13> 2019-03-14 07:46:57.867132 7f32d2496d00 1 bdev create path /var/lib/ceph/tmp/mnt.cb9gUS/block type kernel -12> 2019-03-14 07:46:57.867148 7f32d2496d00 1 bdev(0x55e194d87c00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open path /var/lib/ceph/tmp/mnt.cb9gUS/block -11> 2019-03-14 07:46:57.867485 7f32d2496d00 1 bdev(0x55e194d87c00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -10> 2019-03-14 07:46:57.875626 7f32d2496d00 1 bluestore(/var/lib/ceph/tmp/mnt.cb9gUS) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -9> 2019-03-14 07:46:57.875668 7f32d2496d00 1 bdev create path /var/lib/ceph/tmp/mnt.cb9gUS/block type kernel -8> 2019-03-14 07:46:57.875670 7f32d2496d00 1 bdev(0x55e194d87e00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open path /var/lib/ceph/tmp/mnt.cb9gUS/block -7> 2019-03-14 07:46:57.876137 7f32d2496d00 1 bdev(0x55e194d87e00 /var/lib/ceph/tmp/mnt.cb9gUS/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -6> 2019-03-14 07:46:57.876147 7f32d2496d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.cb9gUS/block size 3.64TiB -5> 2019-03-14 07:46:57.876155 7f32d2496d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 -4> 2019-03-14 07:46:57.876173 7f32d2496d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 -3> 2019-03-14 07:46:57.876205 7f32d2496d00 1 bluefs mkfs uuid c654e950-3e1e-4629-9489-acbbb7d16212 -2> 2019-03-14 07:46:57.908926 7f32d2496d00 1 stupidalloc 0x0x55e194d7c530 shutdown -1> 2019-03-14 07:46:57.908960 7f32d2496d00 1 bluefs mount 0> 2019-03-14 07:46:57.934114 7f32d2496d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f32d2496d00 time 2019-03-14 07:46:57.930656 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x55e18b70baae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x55e18b685224] 3: (BlueFS::_replay(bool)+0x2e9) [0x55e18b697949] 4: (BlueFS::mount()+0x1cf) [0x55e18b69b67f] 5: (BlueStore::_open_db(bool)+0x1746) [0x55e18b5b61d6] 6: (BlueStore::mkfs()+0xf5d) [0x55e18b5be57d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x55e18b19cd55] 8: (main()+0x114e) [0x55e18b0b286e] 9: (__libc_start_main()+0xf5) [0x7f32cf754725] 10: (_start()+0x29) [0x55e18b154389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:57.937721 7f32d2496d00 -1 *** Caught signal (Aborted) ** in thread 7f32d2496d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x55e18b6ced29] 2: (()+0x10c10) [0x7f32d07a2c10] 3: (gsignal()+0x37) [0x7f32cf768f67] 4: (abort()+0x13a) [0x7f32cf76a33a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x55e18b70bc20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x55e18b685224] 7: (BlueFS::_replay(bool)+0x2e9) [0x55e18b697949] 8: (BlueFS::mount()+0x1cf) [0x55e18b69b67f] 9: (BlueStore::_open_db(bool)+0x1746) [0x55e18b5b61d6] 10: (BlueStore::mkfs()+0xf5d) [0x55e18b5be57d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x55e18b19cd55] 12: (main()+0x114e) [0x55e18b0b286e] 13: (__libc_start_main()+0xf5) [0x7f32cf754725] 14: (_start()+0x29) [0x55e18b154389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2019-03-14 07:46:57.937721 7f32d2496d00 -1 *** Caught signal (Aborted) ** in thread 7f32d2496d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x55e18b6ced29] 2: (()+0x10c10) [0x7f32d07a2c10] 3: (gsignal()+0x37) [0x7f32cf768f67] 4: (abort()+0x13a) [0x7f32cf76a33a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x55e18b70bc20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x55e18b685224] 7: (BlueFS::_replay(bool)+0x2e9) [0x55e18b697949] 8: (BlueFS::mount()+0x1cf) [0x55e18b69b67f] 9: (BlueStore::_open_db(bool)+0x1746) [0x55e18b5b61d6] 10: (BlueStore::mkfs()+0xf5d) [0x55e18b5be57d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x55e18b19cd55] 12: (main()+0x114e) [0x55e18b0b286e] 13: (__libc_start_main()+0xf5) [0x7f32cf754725] 14: (_start()+0x29) [0x55e18b154389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:58.621913 7f9bfeaa6d00 0 set uid:gid to 167:167 (ceph:ceph) 2019-03-14 07:46:58.621937 7f9bfeaa6d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124814 2019-03-14 07:46:58.625128 7f9bfeaa6d00 1 bluestore(/var/lib/ceph/tmp/mnt.K31Gy6) mkfs path /var/lib/ceph/tmp/mnt.K31Gy6 2019-03-14 07:46:58.626182 7f9bfeaa6d00 1 bdev create path /var/lib/ceph/tmp/mnt.K31Gy6/block type kernel 2019-03-14 07:46:58.626198 7f9bfeaa6d00 1 bdev(0x560d4de07c00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open path /var/lib/ceph/tmp/mnt.K31Gy6/block 2019-03-14 07:46:58.626515 7f9bfeaa6d00 1 bdev(0x560d4de07c00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:58.633835 7f9bfeaa6d00 1 bluestore(/var/lib/ceph/tmp/mnt.K31Gy6) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-03-14 07:46:58.633882 7f9bfeaa6d00 1 bdev create path /var/lib/ceph/tmp/mnt.K31Gy6/block type kernel 2019-03-14 07:46:58.633884 7f9bfeaa6d00 1 bdev(0x560d4de07e00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open path /var/lib/ceph/tmp/mnt.K31Gy6/block 2019-03-14 07:46:58.634295 7f9bfeaa6d00 1 bdev(0x560d4de07e00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:58.634304 7f9bfeaa6d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.K31Gy6/block size 3.64TiB 2019-03-14 07:46:58.634314 7f9bfeaa6d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 2019-03-14 07:46:58.634333 7f9bfeaa6d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 2019-03-14 07:46:58.634383 7f9bfeaa6d00 1 bluefs mkfs uuid cb617647-cb0d-44ed-8a6b-7677e5985585 2019-03-14 07:46:58.658799 7f9bfeaa6d00 1 stupidalloc 0x0x560d4ddfc530 shutdown 2019-03-14 07:46:58.658833 7f9bfeaa6d00 1 bluefs mount 2019-03-14 07:46:58.682641 7f9bfeaa6d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f9bfeaa6d00 time 2019-03-14 07:46:58.679457 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x560d446aeaae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x560d44628224] 3: (BlueFS::_replay(bool)+0x2e9) [0x560d4463a949] 4: (BlueFS::mount()+0x1cf) [0x560d4463e67f] 5: (BlueStore::_open_db(bool)+0x1746) [0x560d445591d6] 6: (BlueStore::mkfs()+0xf5d) [0x560d4456157d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x560d4413fd55] 8: (main()+0x114e) [0x560d4405586e] 9: (__libc_start_main()+0xf5) [0x7f9bfbd64725] 10: (_start()+0x29) [0x560d440f7389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -57> 2019-03-14 07:46:58.614636 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perfcounters_dump hook 0x560d4ddbe190 -56> 2019-03-14 07:46:58.614648 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command 1 hook 0x560d4ddbe190 -55> 2019-03-14 07:46:58.614650 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perf dump hook 0x560d4ddbe190 -54> 2019-03-14 07:46:58.614658 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perfcounters_schema hook 0x560d4ddbe190 -53> 2019-03-14 07:46:58.614659 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perf histogram dump hook 0x560d4ddbe190 -52> 2019-03-14 07:46:58.614661 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command 2 hook 0x560d4ddbe190 -51> 2019-03-14 07:46:58.614663 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perf schema hook 0x560d4ddbe190 -50> 2019-03-14 07:46:58.614666 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perf histogram schema hook 0x560d4ddbe190 -49> 2019-03-14 07:46:58.614670 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command perf reset hook 0x560d4ddbe190 -48> 2019-03-14 07:46:58.614674 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command config show hook 0x560d4ddbe190 -47> 2019-03-14 07:46:58.614675 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command config help hook 0x560d4ddbe190 -46> 2019-03-14 07:46:58.614679 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command config set hook 0x560d4ddbe190 -45> 2019-03-14 07:46:58.614680 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command config get hook 0x560d4ddbe190 -44> 2019-03-14 07:46:58.614684 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command config diff hook 0x560d4ddbe190 -43> 2019-03-14 07:46:58.614685 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command config diff get hook 0x560d4ddbe190 -42> 2019-03-14 07:46:58.614689 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command log flush hook 0x560d4ddbe190 -41> 2019-03-14 07:46:58.614690 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command log dump hook 0x560d4ddbe190 -40> 2019-03-14 07:46:58.614694 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command log reopen hook 0x560d4ddbe190 -39> 2019-03-14 07:46:58.614701 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command dump_mempools hook 0x560d4dfdd4a8 -38> 2019-03-14 07:46:58.621913 7f9bfeaa6d00 0 set uid:gid to 167:167 (ceph:ceph) -37> 2019-03-14 07:46:58.621937 7f9bfeaa6d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124814 -36> 2019-03-14 07:46:58.622011 7f9bfeaa6d00 5 object store type is bluestore -35> 2019-03-14 07:46:58.624156 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) init /var/run/ceph/ceph-osd.45.asok -34> 2019-03-14 07:46:58.624170 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) bind_and_listen /var/run/ceph/ceph-osd.45.asok -33> 2019-03-14 07:46:58.624233 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command 0 hook 0x560d4ddbc180 -32> 2019-03-14 07:46:58.624240 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command version hook 0x560d4ddbc180 -31> 2019-03-14 07:46:58.624245 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command git_version hook 0x560d4ddbc180 -30> 2019-03-14 07:46:58.624249 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command help hook 0x560d4ddbe260 -29> 2019-03-14 07:46:58.624253 7f9bfeaa6d00 5 asok(0x560d4de1e1c0) register_command get_command_descriptions hook 0x560d4ddbe270 -28> 2019-03-14 07:46:58.624305 7f9bfeaa6d00 10 monclient: build_initial_monmap -27> 2019-03-14 07:46:58.624322 7f9bf92c9700 5 asok(0x560d4de1e1c0) entry start -26> 2019-03-14 07:46:58.624337 7f9bfeaa6d00 10 monclient: get_monmap_privately -25> 2019-03-14 07:46:58.624806 7f9bf8ac8700 2 Event(0x560d4de07080 nevent=5000 time_id=1).set_owner idx=0 owner=140307868714752 -24> 2019-03-14 07:46:58.624830 7f9bf82c7700 2 Event(0x560d4de07480 nevent=5000 time_id=1).set_owner idx=1 owner=140307860322048 -23> 2019-03-14 07:46:58.624848 7f9bf7ac6700 2 Event(0x560d4de07880 nevent=5000 time_id=1).set_owner idx=2 owner=140307851929344 -22> 2019-03-14 07:46:58.624893 7f9bfeaa6d00 1 Processor -- start -21> 2019-03-14 07:46:58.624951 7f9bfeaa6d00 1 -- - start start -20> 2019-03-14 07:46:58.624958 7f9bfeaa6d00 10 monclient: have 1 fsid 04d7f947-8bbd-3880-9d65-252b11e4617c -19> 2019-03-14 07:46:58.624984 7f9bfeaa6d00 1 -- - shutdown_connections -18> 2019-03-14 07:46:58.625062 7f9bfeaa6d00 1 -- - shutdown_connections -17> 2019-03-14 07:46:58.625083 7f9bfeaa6d00 1 -- - wait complete. -16> 2019-03-14 07:46:58.625087 7f9bfeaa6d00 1 -- - >> - conn(0x560d4e178000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -15> 2019-03-14 07:46:58.625099 7f9bfeaa6d00 2 -- - >> - conn(0x560d4e178000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -14> 2019-03-14 07:46:58.625128 7f9bfeaa6d00 1 bluestore(/var/lib/ceph/tmp/mnt.K31Gy6) mkfs path /var/lib/ceph/tmp/mnt.K31Gy6 -13> 2019-03-14 07:46:58.626182 7f9bfeaa6d00 1 bdev create path /var/lib/ceph/tmp/mnt.K31Gy6/block type kernel -12> 2019-03-14 07:46:58.626198 7f9bfeaa6d00 1 bdev(0x560d4de07c00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open path /var/lib/ceph/tmp/mnt.K31Gy6/block -11> 2019-03-14 07:46:58.626515 7f9bfeaa6d00 1 bdev(0x560d4de07c00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -10> 2019-03-14 07:46:58.633835 7f9bfeaa6d00 1 bluestore(/var/lib/ceph/tmp/mnt.K31Gy6) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -9> 2019-03-14 07:46:58.633882 7f9bfeaa6d00 1 bdev create path /var/lib/ceph/tmp/mnt.K31Gy6/block type kernel -8> 2019-03-14 07:46:58.633884 7f9bfeaa6d00 1 bdev(0x560d4de07e00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open path /var/lib/ceph/tmp/mnt.K31Gy6/block -7> 2019-03-14 07:46:58.634295 7f9bfeaa6d00 1 bdev(0x560d4de07e00 /var/lib/ceph/tmp/mnt.K31Gy6/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -6> 2019-03-14 07:46:58.634304 7f9bfeaa6d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.K31Gy6/block size 3.64TiB -5> 2019-03-14 07:46:58.634314 7f9bfeaa6d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 -4> 2019-03-14 07:46:58.634333 7f9bfeaa6d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 -3> 2019-03-14 07:46:58.634383 7f9bfeaa6d00 1 bluefs mkfs uuid cb617647-cb0d-44ed-8a6b-7677e5985585 -2> 2019-03-14 07:46:58.658799 7f9bfeaa6d00 1 stupidalloc 0x0x560d4ddfc530 shutdown -1> 2019-03-14 07:46:58.658833 7f9bfeaa6d00 1 bluefs mount 0> 2019-03-14 07:46:58.682641 7f9bfeaa6d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f9bfeaa6d00 time 2019-03-14 07:46:58.679457 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x560d446aeaae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x560d44628224] 3: (BlueFS::_replay(bool)+0x2e9) [0x560d4463a949] 4: (BlueFS::mount()+0x1cf) [0x560d4463e67f] 5: (BlueStore::_open_db(bool)+0x1746) [0x560d445591d6] 6: (BlueStore::mkfs()+0xf5d) [0x560d4456157d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x560d4413fd55] 8: (main()+0x114e) [0x560d4405586e] 9: (__libc_start_main()+0xf5) [0x7f9bfbd64725] 10: (_start()+0x29) [0x560d440f7389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:58.686448 7f9bfeaa6d00 -1 *** Caught signal (Aborted) ** in thread 7f9bfeaa6d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x560d44671d29] 2: (()+0x10c10) [0x7f9bfcdb2c10] 3: (gsignal()+0x37) [0x7f9bfbd78f67] 4: (abort()+0x13a) [0x7f9bfbd7a33a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x560d446aec20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x560d44628224] 7: (BlueFS::_replay(bool)+0x2e9) [0x560d4463a949] 8: (BlueFS::mount()+0x1cf) [0x560d4463e67f] 9: (BlueStore::_open_db(bool)+0x1746) [0x560d445591d6] 10: (BlueStore::mkfs()+0xf5d) [0x560d4456157d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x560d4413fd55] 12: (main()+0x114e) [0x560d4405586e] 13: (__libc_start_main()+0xf5) [0x7f9bfbd64725] 14: (_start()+0x29) [0x560d440f7389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2019-03-14 07:46:58.686448 7f9bfeaa6d00 -1 *** Caught signal (Aborted) ** in thread 7f9bfeaa6d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x560d44671d29] 2: (()+0x10c10) [0x7f9bfcdb2c10] 3: (gsignal()+0x37) [0x7f9bfbd78f67] 4: (abort()+0x13a) [0x7f9bfbd7a33a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x560d446aec20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x560d44628224] 7: (BlueFS::_replay(bool)+0x2e9) [0x560d4463a949] 8: (BlueFS::mount()+0x1cf) [0x560d4463e67f] 9: (BlueStore::_open_db(bool)+0x1746) [0x560d445591d6] 10: (BlueStore::mkfs()+0xf5d) [0x560d4456157d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x560d4413fd55] 12: (main()+0x114e) [0x560d4405586e] 13: (__libc_start_main()+0xf5) [0x7f9bfbd64725] 14: (_start()+0x29) [0x560d440f7389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:59.893855 7fe7207f1d00 0 set uid:gid to 167:167 (ceph:ceph) 2019-03-14 07:46:59.893868 7fe7207f1d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124881 2019-03-14 07:46:59.896594 7fe7207f1d00 1 bluestore(/var/lib/ceph/tmp/mnt.xxKRKz) mkfs path /var/lib/ceph/tmp/mnt.xxKRKz 2019-03-14 07:46:59.897317 7fe7207f1d00 1 bdev create path /var/lib/ceph/tmp/mnt.xxKRKz/block type kernel 2019-03-14 07:46:59.897330 7fe7207f1d00 1 bdev(0x558ec3497c00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open path /var/lib/ceph/tmp/mnt.xxKRKz/block 2019-03-14 07:46:59.897636 7fe7207f1d00 1 bdev(0x558ec3497c00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:59.900272 7fe7207f1d00 1 bluestore(/var/lib/ceph/tmp/mnt.xxKRKz) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-03-14 07:46:59.900320 7fe7207f1d00 1 bdev create path /var/lib/ceph/tmp/mnt.xxKRKz/block type kernel 2019-03-14 07:46:59.900323 7fe7207f1d00 1 bdev(0x558ec3497e00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open path /var/lib/ceph/tmp/mnt.xxKRKz/block 2019-03-14 07:46:59.900862 7fe7207f1d00 1 bdev(0x558ec3497e00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational 2019-03-14 07:46:59.900888 7fe7207f1d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.xxKRKz/block size 3.64TiB 2019-03-14 07:46:59.900893 7fe7207f1d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 2019-03-14 07:46:59.900912 7fe7207f1d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 2019-03-14 07:46:59.900938 7fe7207f1d00 1 bluefs mkfs uuid 29ab6212-edcf-476e-a6dd-c7ed335330d2 2019-03-14 07:46:59.933574 7fe7207f1d00 1 stupidalloc 0x0x558ec348c530 shutdown 2019-03-14 07:46:59.933592 7fe7207f1d00 1 bluefs mount 2019-03-14 07:46:59.966421 7fe7207f1d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7fe7207f1d00 time 2019-03-14 07:46:59.963674 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x558eb9f5aaae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x558eb9ed4224] 3: (BlueFS::_replay(bool)+0x2e9) [0x558eb9ee6949] 4: (BlueFS::mount()+0x1cf) [0x558eb9eea67f] 5: (BlueStore::_open_db(bool)+0x1746) [0x558eb9e051d6] 6: (BlueStore::mkfs()+0xf5d) [0x558eb9e0d57d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x558eb99ebd55] 8: (main()+0x114e) [0x558eb990186e] 9: (__libc_start_main()+0xf5) [0x7fe71daaf725] 10: (_start()+0x29) [0x558eb99a3389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -57> 2019-03-14 07:46:59.886688 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perfcounters_dump hook 0x558ec344e190 -56> 2019-03-14 07:46:59.886704 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command 1 hook 0x558ec344e190 -55> 2019-03-14 07:46:59.886709 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perf dump hook 0x558ec344e190 -54> 2019-03-14 07:46:59.886715 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perfcounters_schema hook 0x558ec344e190 -53> 2019-03-14 07:46:59.886719 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perf histogram dump hook 0x558ec344e190 -52> 2019-03-14 07:46:59.886722 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command 2 hook 0x558ec344e190 -51> 2019-03-14 07:46:59.886725 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perf schema hook 0x558ec344e190 -50> 2019-03-14 07:46:59.886726 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perf histogram schema hook 0x558ec344e190 -49> 2019-03-14 07:46:59.886730 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command perf reset hook 0x558ec344e190 -48> 2019-03-14 07:46:59.886732 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command config show hook 0x558ec344e190 -47> 2019-03-14 07:46:59.886735 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command config help hook 0x558ec344e190 -46> 2019-03-14 07:46:59.886737 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command config set hook 0x558ec344e190 -45> 2019-03-14 07:46:59.886741 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command config get hook 0x558ec344e190 -44> 2019-03-14 07:46:59.886743 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command config diff hook 0x558ec344e190 -43> 2019-03-14 07:46:59.886746 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command config diff get hook 0x558ec344e190 -42> 2019-03-14 07:46:59.886748 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command log flush hook 0x558ec344e190 -41> 2019-03-14 07:46:59.886751 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command log dump hook 0x558ec344e190 -40> 2019-03-14 07:46:59.886754 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command log reopen hook 0x558ec344e190 -39> 2019-03-14 07:46:59.886762 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command dump_mempools hook 0x558ec366d4a8 -38> 2019-03-14 07:46:59.893855 7fe7207f1d00 0 set uid:gid to 167:167 (ceph:ceph) -37> 2019-03-14 07:46:59.893868 7fe7207f1d00 0 ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable), process ceph-osd, pid 124881 -36> 2019-03-14 07:46:59.893905 7fe7207f1d00 5 object store type is bluestore -35> 2019-03-14 07:46:59.895615 7fe7207f1d00 5 asok(0x558ec34ae1c0) init /var/run/ceph/ceph-osd.45.asok -34> 2019-03-14 07:46:59.895630 7fe7207f1d00 5 asok(0x558ec34ae1c0) bind_and_listen /var/run/ceph/ceph-osd.45.asok -33> 2019-03-14 07:46:59.895684 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command 0 hook 0x558ec344c180 -32> 2019-03-14 07:46:59.895689 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command version hook 0x558ec344c180 -31> 2019-03-14 07:46:59.895692 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command git_version hook 0x558ec344c180 -30> 2019-03-14 07:46:59.895698 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command help hook 0x558ec344e260 -29> 2019-03-14 07:46:59.895701 7fe7207f1d00 5 asok(0x558ec34ae1c0) register_command get_command_descriptions hook 0x558ec344e270 -28> 2019-03-14 07:46:59.895755 7fe7207f1d00 10 monclient: build_initial_monmap -27> 2019-03-14 07:46:59.895755 7fe71b014700 5 asok(0x558ec34ae1c0) entry start -26> 2019-03-14 07:46:59.895785 7fe7207f1d00 10 monclient: get_monmap_privately -25> 2019-03-14 07:46:59.896215 7fe71a813700 2 Event(0x558ec3497080 nevent=5000 time_id=1).set_owner idx=0 owner=140630558848768 -24> 2019-03-14 07:46:59.896282 7fe71a012700 2 Event(0x558ec3497480 nevent=5000 time_id=1).set_owner idx=1 owner=140630550456064 -23> 2019-03-14 07:46:59.896288 7fe719811700 2 Event(0x558ec3497880 nevent=5000 time_id=1).set_owner idx=2 owner=140630542063360 -22> 2019-03-14 07:46:59.896348 7fe7207f1d00 1 Processor -- start -21> 2019-03-14 07:46:59.896415 7fe7207f1d00 1 -- - start start -20> 2019-03-14 07:46:59.896422 7fe7207f1d00 10 monclient: have 1 fsid 04d7f947-8bbd-3880-9d65-252b11e4617c -19> 2019-03-14 07:46:59.896448 7fe7207f1d00 1 -- - shutdown_connections -18> 2019-03-14 07:46:59.896525 7fe7207f1d00 1 -- - shutdown_connections -17> 2019-03-14 07:46:59.896546 7fe7207f1d00 1 -- - wait complete. -16> 2019-03-14 07:46:59.896549 7fe7207f1d00 1 -- - >> - conn(0x558ec3804000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down -15> 2019-03-14 07:46:59.896563 7fe7207f1d00 2 -- - >> - conn(0x558ec3804000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop -14> 2019-03-14 07:46:59.896594 7fe7207f1d00 1 bluestore(/var/lib/ceph/tmp/mnt.xxKRKz) mkfs path /var/lib/ceph/tmp/mnt.xxKRKz -13> 2019-03-14 07:46:59.897317 7fe7207f1d00 1 bdev create path /var/lib/ceph/tmp/mnt.xxKRKz/block type kernel -12> 2019-03-14 07:46:59.897330 7fe7207f1d00 1 bdev(0x558ec3497c00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open path /var/lib/ceph/tmp/mnt.xxKRKz/block -11> 2019-03-14 07:46:59.897636 7fe7207f1d00 1 bdev(0x558ec3497c00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -10> 2019-03-14 07:46:59.900272 7fe7207f1d00 1 bluestore(/var/lib/ceph/tmp/mnt.xxKRKz) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -9> 2019-03-14 07:46:59.900320 7fe7207f1d00 1 bdev create path /var/lib/ceph/tmp/mnt.xxKRKz/block type kernel -8> 2019-03-14 07:46:59.900323 7fe7207f1d00 1 bdev(0x558ec3497e00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open path /var/lib/ceph/tmp/mnt.xxKRKz/block -7> 2019-03-14 07:46:59.900862 7fe7207f1d00 1 bdev(0x558ec3497e00 /var/lib/ceph/tmp/mnt.xxKRKz/block) open size 4000681103360 (0x3a37b2d1000, 3.64TiB) block_size 4096 (4KiB) rotational -6> 2019-03-14 07:46:59.900888 7fe7207f1d00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/tmp/mnt.xxKRKz/block size 3.64TiB -5> 2019-03-14 07:46:59.900893 7fe7207f1d00 1 bluefs add_block_extent bdev 1 0x1bf1c600000~2542600000 -4> 2019-03-14 07:46:59.900912 7fe7207f1d00 1 bluefs mkfs osd_uuid d3a09a51-6b66-4464-8e81-126acc818d62 -3> 2019-03-14 07:46:59.900938 7fe7207f1d00 1 bluefs mkfs uuid 29ab6212-edcf-476e-a6dd-c7ed335330d2 -2> 2019-03-14 07:46:59.933574 7fe7207f1d00 1 stupidalloc 0x0x558ec348c530 shutdown -1> 2019-03-14 07:46:59.933592 7fe7207f1d00 1 bluefs mount 0> 2019-03-14 07:46:59.966421 7fe7207f1d00 -1 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7fe7207f1d00 time 2019-03-14 07:46:59.963674 /home/abuild/rpmbuild/BUILD/ceph-12.2.10-551-gbb089269ea/src/os/bluestore/BlueFS.cc: 976: FAILED assert(r == 0) ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x558eb9f5aaae] 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x558eb9ed4224] 3: (BlueFS::_replay(bool)+0x2e9) [0x558eb9ee6949] 4: (BlueFS::mount()+0x1cf) [0x558eb9eea67f] 5: (BlueStore::_open_db(bool)+0x1746) [0x558eb9e051d6] 6: (BlueStore::mkfs()+0xf5d) [0x558eb9e0d57d] 7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x558eb99ebd55] 8: (main()+0x114e) [0x558eb990186e] 9: (__libc_start_main()+0xf5) [0x7fe71daaf725] 10: (_start()+0x29) [0x558eb99a3389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events --- 2019-03-14 07:46:59.969607 7fe7207f1d00 -1 *** Caught signal (Aborted) ** in thread 7fe7207f1d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x558eb9f1dd29] 2: (()+0x10c10) [0x7fe71eafdc10] 3: (gsignal()+0x37) [0x7fe71dac3f67] 4: (abort()+0x13a) [0x7fe71dac533a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x558eb9f5ac20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x558eb9ed4224] 7: (BlueFS::_replay(bool)+0x2e9) [0x558eb9ee6949] 8: (BlueFS::mount()+0x1cf) [0x558eb9eea67f] 9: (BlueStore::_open_db(bool)+0x1746) [0x558eb9e051d6] 10: (BlueStore::mkfs()+0xf5d) [0x558eb9e0d57d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x558eb99ebd55] 12: (main()+0x114e) [0x558eb990186e] 13: (__libc_start_main()+0xf5) [0x7fe71daaf725] 14: (_start()+0x29) [0x558eb99a3389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2019-03-14 07:46:59.969607 7fe7207f1d00 -1 *** Caught signal (Aborted) ** in thread 7fe7207f1d00 thread_name:ceph-osd ceph version 12.2.10-551-gbb089269ea (bb089269ea0c1272294c6b9777123ac81662b6d2) luminous (stable) 1: (()+0xa66d29) [0x558eb9f1dd29] 2: (()+0x10c10) [0x7fe71eafdc10] 3: (gsignal()+0x37) [0x7fe71dac3f67] 4: (abort()+0x13a) [0x7fe71dac533a] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x280) [0x558eb9f5ac20] 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xcb4) [0x558eb9ed4224] 7: (BlueFS::_replay(bool)+0x2e9) [0x558eb9ee6949] 8: (BlueFS::mount()+0x1cf) [0x558eb9eea67f] 9: (BlueStore::_open_db(bool)+0x1746) [0x558eb9e051d6] 10: (BlueStore::mkfs()+0xf5d) [0x558eb9e0d57d] 11: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x558eb99ebd55] 12: (main()+0x114e) [0x558eb990186e] 13: (__libc_start_main()+0xf5) [0x7fe71daaf725] 14: (_start()+0x29) [0x558eb99a3389] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.45.log --- end dump of recent events ---