Feb 04 09:32:12 osd07 systemd[1]: Starting Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e... Feb 04 09:32:12 osd07 docker[30561]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:12 osd07 docker[30561]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:12 osd07 systemd[1]: Started Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:12 osd07 bash[30576]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:12 osd07 bash[30576]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-activate Feb 04 09:32:12 osd07 bash[30576]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:13 osd07 bash[30576]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:13 osd07 bash[30576]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 --path /var/lib/ceph/osd/ceph-18 --no-mon-config Feb 04 09:32:13 osd07 bash[30576]: Running command: /usr/bin/ln -snf /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:13 osd07 bash[30576]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:13 osd07 bash[30576]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7 Feb 04 09:32:13 osd07 bash[30576]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:13 osd07 bash[30576]: --> ceph-volume lvm activate successful for osd ID: 18 Feb 04 09:32:13 osd07 bash[30576]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:13 osd07 bash[30576]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:13 osd07 bash[30576]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.025+0000 7f0108390f40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.025+0000 7f0108390f40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.025+0000 7f0108390f40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f00ff904700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f00ff103700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f00fe902700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev aio_wait 0x563972d92bb0 done Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.281+0000 7f00ff103700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.321+0000 7f00fe902700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.321+0000 7f00fe902700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.321+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.529+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f00ff904700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f00ff904700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f0108390f40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.561+0000 7f0108390f40 -1 Falling back to public interface Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 0 load: jerasure load: lrc load: isa Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f00fe902700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.825+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f00fe902700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f00fe902700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f00fb88b700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f00fb08a700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f00fa889700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f0108390f40 time 2021-02-04T08:32:14.872983+0000 Feb 04 09:32:14 osd07 bash[30576]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluefs mount Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluefs _open_super Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f0108390f40 5 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.869+0000 7f00fa088700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5639688b4372] Feb 04 09:32:14 osd07 bash[30576]: 2: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 4: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 5: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 10: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 11: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 12: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 13: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.877+0000 7f0108390f40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f0108390f40 time 2021-02-04T08:32:14.872983+0000 Feb 04 09:32:14 osd07 bash[30576]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5639688b4372] Feb 04 09:32:14 osd07 bash[30576]: 2: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 4: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 5: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 10: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 11: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 12: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 13: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: *** Caught signal (Aborted) ** Feb 04 09:32:14 osd07 bash[30576]: in thread 7f0108390f40 thread_name:ceph-osd Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (()+0x12b20) [0x7f01060f8b20] Feb 04 09:32:14 osd07 bash[30576]: 2: (gsignal()+0x10f) [0x7f0104d607ff] Feb 04 09:32:14 osd07 bash[30576]: 3: (abort()+0x127) [0x7f0104d4ac35] Feb 04 09:32:14 osd07 bash[30576]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5639688b43c3] Feb 04 09:32:14 osd07 bash[30576]: 5: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 13: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 14: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 15: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 16: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: debug 2021-02-04T08:32:14.881+0000 7f0108390f40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:14 osd07 bash[30576]: in thread 7f0108390f40 thread_name:ceph-osd Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (()+0x12b20) [0x7f01060f8b20] Feb 04 09:32:14 osd07 bash[30576]: 2: (gsignal()+0x10f) [0x7f0104d607ff] Feb 04 09:32:14 osd07 bash[30576]: 3: (abort()+0x127) [0x7f0104d4ac35] Feb 04 09:32:14 osd07 bash[30576]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5639688b43c3] Feb 04 09:32:14 osd07 bash[30576]: 5: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 13: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 14: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 15: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 16: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:14 osd07 bash[30576]: --- begin dump of recent events --- Feb 04 09:32:14 osd07 bash[30576]: debug -378> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command assert hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -377> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command abort hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -376> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command leak_some_memory hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -375> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perfcounters_dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -374> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command 1 hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -373> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -372> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perfcounters_schema hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -371> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf histogram dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -370> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command 2 hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -369> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf schema hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -368> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf histogram schema hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -367> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf reset hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -366> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config show hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -365> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config help hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -364> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config set hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -363> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config unset hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -362> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config get hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -361> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config diff hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -360> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config diff get hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -359> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command injectargs hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -358> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command log flush hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -357> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command log dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -356> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command log reopen hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -355> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command dump_mempools hook 0x5639739de068 Feb 04 09:32:14 osd07 bash[30576]: debug -354> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: get_monmap_and_config Feb 04 09:32:14 osd07 bash[30576]: debug -353> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: build_initial_monmap Feb 04 09:32:14 osd07 bash[30576]: debug -352> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: monmap: Feb 04 09:32:14 osd07 bash[30576]: epoch 0 Feb 04 09:32:14 osd07 bash[30576]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:14 osd07 bash[30576]: last_changed 2021-02-04T08:32:14.024507+0000 Feb 04 09:32:14 osd07 bash[30576]: created 2021-02-04T08:32:14.024507+0000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 0 (unknown) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:14 osd07 bash[30576]: debug -351> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -350> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -349> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -348> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -347> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -346> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -345> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -344> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -343> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -342> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -341> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -340> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -339> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -338> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -337> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -336> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -335> 2021-02-04T08:32:14.021+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -334> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: init Feb 04 09:32:14 osd07 bash[30576]: debug -333> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -332> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -331> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -330> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -329> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -328> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -327> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -326> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -325> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -324> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -323> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -322> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -321> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -320> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -319> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -318> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -317> 2021-02-04T08:32:14.021+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -316> 2021-02-04T08:32:14.021+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -315> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: _reopen_session rank -1 Feb 04 09:32:14 osd07 bash[30576]: debug -314> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: _add_conns ranks=[0,1,2] Feb 04 09:32:14 osd07 bash[30576]: debug -313> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): picked mon.noname-a con 0x563972e08c00 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:14 osd07 bash[30576]: debug -312> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): picked mon.noname-b con 0x563972e09000 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:14 osd07 bash[30576]: debug -311> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): picked mon.noname-c con 0x563972e09400 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:14 osd07 bash[30576]: debug -310> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:14 osd07 bash[30576]: debug -309> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:14 osd07 bash[30576]: debug -308> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:14 osd07 bash[30576]: debug -307> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): _renew_subs Feb 04 09:32:14 osd07 bash[30576]: debug -306> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:14.025908+0000 Feb 04 09:32:14 osd07 bash[30576]: debug -305> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): get_auth_request con 0x563972e09000 auth_method 0 Feb 04 09:32:14 osd07 bash[30576]: debug -304> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:14 osd07 bash[30576]: debug -303> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:14 osd07 bash[30576]: debug -302> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): get_auth_request con 0x563972e09400 auth_method 0 Feb 04 09:32:14 osd07 bash[30576]: debug -301> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:14 osd07 bash[30576]: debug -300> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:14 osd07 bash[30576]: debug -299> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): get_auth_request con 0x563972e08c00 auth_method 0 Feb 04 09:32:14 osd07 bash[30576]: debug -298> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:14 osd07 bash[30576]: debug -297> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:14 osd07 bash[30576]: debug -296> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:14 osd07 bash[30576]: debug -295> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:14 osd07 bash[30576]: debug -294> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:14 osd07 bash[30576]: debug -293> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:14 osd07 bash[30576]: debug -292> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:14 osd07 bash[30576]: debug -291> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:14 osd07 bash[30576]: debug -290> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:14 osd07 bash[30576]: debug -289> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:14 osd07 bash[30576]: debug -288> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:14 osd07 bash[30576]: debug -287> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_done global_id 3644906 payload 450 Feb 04 09:32:14 osd07 bash[30576]: debug -286> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient: _finish_hunting 0 Feb 04 09:32:14 osd07 bash[30576]: debug -285> 2021-02-04T08:32:14.025+0000 7f0101908700 1 monclient: found mon.noname-b Feb 04 09:32:14 osd07 bash[30576]: debug -284> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.15:3300/0 Feb 04 09:32:14 osd07 bash[30576]: debug -283> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:14 osd07 bash[30576]: debug -282> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: got monmap 9 from mon.noname-b (according to old e9) Feb 04 09:32:14 osd07 bash[30576]: debug -281> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: dump: Feb 04 09:32:14 osd07 bash[30576]: epoch 9 Feb 04 09:32:14 osd07 bash[30576]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:14 osd07 bash[30576]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:14 osd07 bash[30576]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 15 (octopus) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:14 osd07 bash[30576]: debug -280> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: _finish_auth 0 Feb 04 09:32:14 osd07 bash[30576]: debug -279> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:31:44.028550+0000) Feb 04 09:32:14 osd07 bash[30576]: debug -278> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: _send_mon_message to mon.osd05 at v2:10.10.0.15:3300/0 Feb 04 09:32:14 osd07 bash[30576]: debug -277> 2021-02-04T08:32:14.025+0000 7f0108390f40 5 monclient: authenticate success, global_id 3644906 Feb 04 09:32:14 osd07 bash[30576]: debug -276> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:14 osd07 bash[30576]: debug -275> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: get_monmap_and_config success Feb 04 09:32:14 osd07 bash[30576]: debug -274> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: shutdown Feb 04 09:32:14 osd07 bash[30576]: debug -273> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:14 osd07 bash[30576]: debug -272> 2021-02-04T08:32:14.025+0000 7f00ff904700 4 set_mon_vals no callback set Feb 04 09:32:14 osd07 bash[30576]: debug -271> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: got monmap 9 from mon.osd05 (according to old e9) Feb 04 09:32:14 osd07 bash[30576]: debug -270> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:14 osd07 bash[30576]: debug -269> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:14 osd07 bash[30576]: debug -268> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: dump: Feb 04 09:32:14 osd07 bash[30576]: epoch 9 Feb 04 09:32:14 osd07 bash[30576]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:14 osd07 bash[30576]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:14 osd07 bash[30576]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 15 (octopus) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:14 osd07 bash[30576]: debug -267> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:14 osd07 bash[30576]: debug -266> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:14 osd07 bash[30576]: debug -265> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:14 osd07 bash[30576]: debug -264> 2021-02-04T08:32:14.025+0000 7f0108390f40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:14 osd07 bash[30576]: debug -263> 2021-02-04T08:32:14.025+0000 7f0108390f40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:14 osd07 bash[30576]: debug -262> 2021-02-04T08:32:14.025+0000 7f0108390f40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:14 osd07 bash[30576]: debug -261> 2021-02-04T08:32:14.025+0000 7f0108390f40 5 asok(0x563972e0a000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:14 osd07 bash[30576]: debug -260> 2021-02-04T08:32:14.025+0000 7f0108390f40 5 asok(0x563972e0a000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:14 osd07 bash[30576]: debug -259> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command 0 hook 0x563972d62388 Feb 04 09:32:14 osd07 bash[30576]: debug -258> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command version hook 0x563972d62388 Feb 04 09:32:14 osd07 bash[30576]: debug -257> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command git_version hook 0x563972d62388 Feb 04 09:32:14 osd07 bash[30576]: debug -256> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command help hook 0x563972d5e370 Feb 04 09:32:14 osd07 bash[30576]: debug -255> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command get_command_descriptions hook 0x563972d5e070 Feb 04 09:32:14 osd07 bash[30576]: debug -254> 2021-02-04T08:32:14.029+0000 7f0100906700 5 asok(0x563972e0a000) entry start Feb 04 09:32:14 osd07 bash[30576]: debug -253> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 object store type is bluestore Feb 04 09:32:14 osd07 bash[30576]: debug -252> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:14 osd07 bash[30576]: debug -251> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -250> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -249> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -248> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -247> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -246> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -245> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -244> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -243> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -242> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -241> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -240> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -239> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:14 osd07 bash[30576]: debug -238> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -237> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -236> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -235> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -234> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -233> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -232> 2021-02-04T08:32:14.029+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -231> 2021-02-04T08:32:14.029+0000 7f00ff904700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -230> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -229> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug -228> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluestore bluefs available hook 0x563972d5e0b0 Feb 04 09:32:14 osd07 bash[30576]: debug -227> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluefs stats hook 0x563972d5e0b0 Feb 04 09:32:14 osd07 bash[30576]: debug -226> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -225> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -224> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -223> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -222> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -221> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -220> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -219> 2021-02-04T08:32:14.029+0000 7f00ff103700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -218> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -217> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:14 osd07 bash[30576]: debug -216> 2021-02-04T08:32:14.029+0000 7f00fe902700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -215> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) unregister_commands bluefs stats Feb 04 09:32:14 osd07 bash[30576]: debug -214> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) unregister_commands bluestore bluefs available Feb 04 09:32:14 osd07 bash[30576]: debug -213> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev aio_wait 0x563972d92bb0 done Feb 04 09:32:14 osd07 bash[30576]: debug -212> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug -211> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug -210> 2021-02-04T08:32:14.281+0000 7f00ff103700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug -209> 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug -208> 2021-02-04T08:32:14.321+0000 7f00fe902700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug -207> 2021-02-04T08:32:14.321+0000 7f00fe902700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug -206> 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug -205> 2021-02-04T08:32:14.321+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug -204> 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug -203> 2021-02-04T08:32:14.529+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug -202> 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug -201> 2021-02-04T08:32:14.561+0000 7f00ff904700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug -200> 2021-02-04T08:32:14.561+0000 7f00ff904700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug -199> 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug -198> 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:14 osd07 bash[30576]: debug -197> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -196> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -195> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -194> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -193> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -192> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -191> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -190> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -189> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -188> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -187> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -186> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -185> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -184> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -183> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -182> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -181> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -180> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -179> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -178> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -177> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -176> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -175> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -174> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -173> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -172> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -171> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -170> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -169> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -168> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -167> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -166> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -165> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -164> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -163> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -162> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -161> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -160> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -159> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -158> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -157> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -156> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -155> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -154> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -153> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -152> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -151> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -150> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -149> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -148> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -147> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -146> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -145> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -144> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -143> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -142> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -141> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -140> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -139> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -138> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -137> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -136> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -135> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -134> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -133> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -132> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -131> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -130> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -129> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -128> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -127> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -126> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -125> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -124> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -123> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -122> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -121> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -120> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -119> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -118> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -117> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -116> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -115> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -114> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -113> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -112> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -111> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -110> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -109> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -108> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -107> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -106> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -105> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -104> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -103> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -102> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -101> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -100> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -99> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -98> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -97> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -96> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -95> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -94> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -93> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -92> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -91> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -90> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -89> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -88> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -87> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -86> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -85> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -84> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -83> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -82> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -81> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -80> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -79> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -78> 2021-02-04T08:32:14.561+0000 7f0108390f40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:14 osd07 bash[30576]: debug -77> 2021-02-04T08:32:14.561+0000 7f0108390f40 -1 Falling back to public interface Feb 04 09:32:14 osd07 bash[30576]: debug -76> 2021-02-04T08:32:14.565+0000 7f0108390f40 10 monclient: build_initial_monmap Feb 04 09:32:14 osd07 bash[30576]: debug -75> 2021-02-04T08:32:14.565+0000 7f0108390f40 10 monclient: monmap: Feb 04 09:32:14 osd07 bash[30576]: epoch 0 Feb 04 09:32:14 osd07 bash[30576]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:14 osd07 bash[30576]: last_changed 0.000000 Feb 04 09:32:14 osd07 bash[30576]: created 0.000000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 0 (unknown) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:14 osd07 bash[30576]: debug -74> 2021-02-04T08:32:14.577+0000 7f0108390f40 0 load: jerasure load: lrc load: isa Feb 04 09:32:14 osd07 bash[30576]: debug -73> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -72> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -71> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:14 osd07 bash[30576]: debug -70> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -69> 2021-02-04T08:32:14.577+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -68> 2021-02-04T08:32:14.577+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -67> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -66> 2021-02-04T08:32:14.577+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -65> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -64> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -63> 2021-02-04T08:32:14.577+0000 7f00fe902700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -62> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -61> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug -60> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug -59> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug -58> 2021-02-04T08:32:14.825+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug -57> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug -56> 2021-02-04T08:32:14.869+0000 7f00fe902700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug -55> 2021-02-04T08:32:14.869+0000 7f00fe902700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug -54> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug -53> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 asok(0x563972e0a000) register_command objecter_requests hook 0x563972d5e980 Feb 04 09:32:14 osd07 bash[30576]: debug -52> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -51> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -50> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -49> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -48> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -47> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -46> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -45> 2021-02-04T08:32:14.869+0000 7f0108390f40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:14 osd07 bash[30576]: debug -44> 2021-02-04T08:32:14.869+0000 7f0108390f40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:14 osd07 bash[30576]: debug -43> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:14 osd07 bash[30576]: debug -42> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:14 osd07 bash[30576]: debug -41> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -40> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -39> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -38> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -37> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:14 osd07 bash[30576]: debug -36> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -35> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -34> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -33> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -32> 2021-02-04T08:32:14.869+0000 7f00fb88b700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -31> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -30> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -29> 2021-02-04T08:32:14.869+0000 7f00fb08a700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -28> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -27> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug -26> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -25> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -24> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -23> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -22> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:14 osd07 bash[30576]: debug -21> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -20> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -19> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:14 osd07 bash[30576]: debug -18> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:14 osd07 bash[30576]: debug -17> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluestore bluefs available hook 0x563972d5e990 Feb 04 09:32:14 osd07 bash[30576]: debug -16> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluefs stats hook 0x563972d5e990 Feb 04 09:32:14 osd07 bash[30576]: debug -15> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -14> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -13> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -12> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -11> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -10> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -9> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -8> 2021-02-04T08:32:14.869+0000 7f00fa889700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -7> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -6> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:14 osd07 bash[30576]: debug -5> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluefs mount Feb 04 09:32:14 osd07 bash[30576]: debug -4> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluefs _open_super Feb 04 09:32:14 osd07 bash[30576]: debug -3> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:14 osd07 bash[30576]: debug -2> 2021-02-04T08:32:14.869+0000 7f00fa088700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -1> 2021-02-04T08:32:14.877+0000 7f0108390f40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f0108390f40 time 2021-02-04T08:32:14.872983+0000 Feb 04 09:32:14 osd07 bash[30576]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5639688b4372] Feb 04 09:32:14 osd07 bash[30576]: 2: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 4: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 5: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 10: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 11: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 12: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 13: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: debug 0> 2021-02-04T08:32:14.881+0000 7f0108390f40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:14 osd07 bash[30576]: in thread 7f0108390f40 thread_name:ceph-osd Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (()+0x12b20) [0x7f01060f8b20] Feb 04 09:32:14 osd07 bash[30576]: 2: (gsignal()+0x10f) [0x7f0104d607ff] Feb 04 09:32:14 osd07 bash[30576]: 3: (abort()+0x127) [0x7f0104d4ac35] Feb 04 09:32:14 osd07 bash[30576]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5639688b43c3] Feb 04 09:32:14 osd07 bash[30576]: 5: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 13: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 14: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 15: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 16: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:14 osd07 bash[30576]: --- logging levels --- Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 none Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 lockdep Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 context Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 crush Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_balancer Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_locker Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_log Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_log_expire Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_migrator Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 buffer Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 timer Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 filer Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 striper Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 objecter Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rados Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd_mirror Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd_replay Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd_rwl Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 journaler Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 objectcacher Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 immutable_obj_cache Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 client Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 osd Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 optracker Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 objclass Feb 04 09:32:14 osd07 bash[30576]: 1/ 3 filestore Feb 04 09:32:14 osd07 bash[30576]: 1/ 3 journal Feb 04 09:32:14 osd07 bash[30576]: 0/ 0 ms Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mon Feb 04 09:32:14 osd07 bash[30576]: 0/10 monc Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 paxos Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 tp Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 auth Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 crypto Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 finisher Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 reserver Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 heartbeatmap Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 perfcounter Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 rgw Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 rgw_sync Feb 04 09:32:14 osd07 bash[30576]: 1/10 civetweb Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 javaclient Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 asok Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 throttle Feb 04 09:32:14 osd07 bash[30576]: 0/ 0 refs Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 compressor Feb 04 09:32:14 osd07 bash[30576]: 20/20 bluestore Feb 04 09:32:14 osd07 bash[30576]: 20/20 bluefs Feb 04 09:32:14 osd07 bash[30576]: 20/20 bdev Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 kstore Feb 04 09:32:14 osd07 bash[30576]: 4/ 5 rocksdb Feb 04 09:32:14 osd07 bash[30576]: 4/ 5 leveldb Feb 04 09:32:14 osd07 bash[30576]: 4/ 5 memdb Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 fuse Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mgr Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mgrc Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 dpdk Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 eventtrace Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 prioritycache Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 test Feb 04 09:32:14 osd07 bash[30576]: -2/-2 (syslog threshold) Feb 04 09:32:14 osd07 bash[30576]: 99/99 (stderr threshold) Feb 04 09:32:14 osd07 bash[30576]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:14 osd07 bash[30576]: 7f00fa088700 / bstore_discard Feb 04 09:32:14 osd07 bash[30576]: 7f00fa889700 / bstore_aio Feb 04 09:32:14 osd07 bash[30576]: 7f00fb08a700 / bstore_discard Feb 04 09:32:14 osd07 bash[30576]: 7f00fb88b700 / bstore_aio Feb 04 09:32:14 osd07 bash[30576]: 7f00fe902700 / OpHistorySvc Feb 04 09:32:14 osd07 bash[30576]: 7f00ff103700 / safe_timer Feb 04 09:32:14 osd07 bash[30576]: 7f00ff904700 / signal_handler Feb 04 09:32:14 osd07 bash[30576]: 7f0100105700 / ceph-osd Feb 04 09:32:14 osd07 bash[30576]: 7f0100906700 / admin_socket Feb 04 09:32:14 osd07 bash[30576]: 7f0101107700 / service Feb 04 09:32:14 osd07 bash[30576]: 7f0101908700 / msgr-worker-2 Feb 04 09:32:14 osd07 bash[30576]: 7f0102109700 / msgr-worker-1 Feb 04 09:32:14 osd07 bash[30576]: 7f010290a700 / msgr-worker-0 Feb 04 09:32:14 osd07 bash[30576]: 7f0108390f40 / ceph-osd Feb 04 09:32:14 osd07 bash[30576]: max_recent 10000 Feb 04 09:32:14 osd07 bash[30576]: max_new 1000 Feb 04 09:32:14 osd07 bash[30576]: log_file Feb 04 09:32:14 osd07 bash[30576]: --- end dump of recent events --- Feb 04 09:32:14 osd07 bash[30576]: --- begin dump of recent events --- Feb 04 09:32:14 osd07 bash[30576]: debug -378> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command assert hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -377> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command abort hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -376> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command leak_some_memory hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -375> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perfcounters_dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -374> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command 1 hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -373> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -372> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perfcounters_schema hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -371> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf histogram dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -370> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command 2 hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -369> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf schema hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -368> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf histogram schema hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -367> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command perf reset hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -366> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config show hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -365> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config help hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -364> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config set hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -363> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config unset hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -362> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config get hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -361> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config diff hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -360> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command config diff get hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -359> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command injectargs hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -358> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command log flush hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -357> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command log dump hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -356> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command log reopen hook 0x563972d5e5b0 Feb 04 09:32:14 osd07 bash[30576]: debug -355> 2021-02-04T08:32:14.017+0000 7f0108390f40 5 asok(0x563972e0a000) register_command dump_mempools hook 0x5639739de068 Feb 04 09:32:14 osd07 bash[30576]: debug -354> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: get_monmap_and_config Feb 04 09:32:14 osd07 bash[30576]: debug -353> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: build_initial_monmap Feb 04 09:32:14 osd07 bash[30576]: debug -352> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: monmap: Feb 04 09:32:14 osd07 bash[30576]: epoch 0 Feb 04 09:32:14 osd07 bash[30576]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:14 osd07 bash[30576]: last_changed 2021-02-04T08:32:14.024507+0000 Feb 04 09:32:14 osd07 bash[30576]: created 2021-02-04T08:32:14.024507+0000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 0 (unknown) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:14 osd07 bash[30576]: debug -351> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -350> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -349> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -348> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -347> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -346> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -345> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -344> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -343> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -342> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -341> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -340> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -339> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -338> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -337> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -336> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x563972df4940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -335> 2021-02-04T08:32:14.021+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -334> 2021-02-04T08:32:14.021+0000 7f0108390f40 10 monclient: init Feb 04 09:32:14 osd07 bash[30576]: debug -333> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -332> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -331> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -330> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -329> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -328> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -327> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -326> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -325> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -324> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -323> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -322> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -321> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -320> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -319> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -318> 2021-02-04T08:32:14.021+0000 7f0108390f40 5 AuthRegistry(0x7fff688091b0) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -317> 2021-02-04T08:32:14.021+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -316> 2021-02-04T08:32:14.021+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -315> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: _reopen_session rank -1 Feb 04 09:32:14 osd07 bash[30576]: debug -314> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: _add_conns ranks=[0,1,2] Feb 04 09:32:14 osd07 bash[30576]: debug -313> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): picked mon.noname-a con 0x563972e08c00 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:14 osd07 bash[30576]: debug -312> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): picked mon.noname-b con 0x563972e09000 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:14 osd07 bash[30576]: debug -311> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): picked mon.noname-c con 0x563972e09400 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:14 osd07 bash[30576]: debug -310> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:14 osd07 bash[30576]: debug -309> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:14 osd07 bash[30576]: debug -308> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:14 osd07 bash[30576]: debug -307> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): _renew_subs Feb 04 09:32:14 osd07 bash[30576]: debug -306> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:14.025908+0000 Feb 04 09:32:14 osd07 bash[30576]: debug -305> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): get_auth_request con 0x563972e09000 auth_method 0 Feb 04 09:32:14 osd07 bash[30576]: debug -304> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:14 osd07 bash[30576]: debug -303> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:14 osd07 bash[30576]: debug -302> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): get_auth_request con 0x563972e09400 auth_method 0 Feb 04 09:32:14 osd07 bash[30576]: debug -301> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:14 osd07 bash[30576]: debug -300> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:14 osd07 bash[30576]: debug -299> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): get_auth_request con 0x563972e08c00 auth_method 0 Feb 04 09:32:14 osd07 bash[30576]: debug -298> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:14 osd07 bash[30576]: debug -297> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:14 osd07 bash[30576]: debug -296> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:14 osd07 bash[30576]: debug -295> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:14 osd07 bash[30576]: debug -294> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:14 osd07 bash[30576]: debug -293> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:14 osd07 bash[30576]: debug -292> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:14 osd07 bash[30576]: debug -291> 2021-02-04T08:32:14.025+0000 7f010290a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:14 osd07 bash[30576]: debug -290> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:14 osd07 bash[30576]: debug -289> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:14 osd07 bash[30576]: debug -288> 2021-02-04T08:32:14.025+0000 7f0102109700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:14 osd07 bash[30576]: debug -287> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient(hunting): handle_auth_done global_id 3644906 payload 450 Feb 04 09:32:14 osd07 bash[30576]: debug -286> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient: _finish_hunting 0 Feb 04 09:32:14 osd07 bash[30576]: debug -285> 2021-02-04T08:32:14.025+0000 7f0101908700 1 monclient: found mon.noname-b Feb 04 09:32:14 osd07 bash[30576]: debug -284> 2021-02-04T08:32:14.025+0000 7f0101908700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.15:3300/0 Feb 04 09:32:14 osd07 bash[30576]: debug -283> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:14 osd07 bash[30576]: debug -282> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: got monmap 9 from mon.noname-b (according to old e9) Feb 04 09:32:14 osd07 bash[30576]: debug -281> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: dump: Feb 04 09:32:14 osd07 bash[30576]: epoch 9 Feb 04 09:32:14 osd07 bash[30576]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:14 osd07 bash[30576]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:14 osd07 bash[30576]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 15 (octopus) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:14 osd07 bash[30576]: debug -280> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: _finish_auth 0 Feb 04 09:32:14 osd07 bash[30576]: debug -279> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:31:44.028550+0000) Feb 04 09:32:14 osd07 bash[30576]: debug -278> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: _send_mon_message to mon.osd05 at v2:10.10.0.15:3300/0 Feb 04 09:32:14 osd07 bash[30576]: debug -277> 2021-02-04T08:32:14.025+0000 7f0108390f40 5 monclient: authenticate success, global_id 3644906 Feb 04 09:32:14 osd07 bash[30576]: debug -276> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:14 osd07 bash[30576]: debug -275> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: get_monmap_and_config success Feb 04 09:32:14 osd07 bash[30576]: debug -274> 2021-02-04T08:32:14.025+0000 7f0108390f40 10 monclient: shutdown Feb 04 09:32:14 osd07 bash[30576]: debug -273> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:14 osd07 bash[30576]: debug -272> 2021-02-04T08:32:14.025+0000 7f00ff904700 4 set_mon_vals no callback set Feb 04 09:32:14 osd07 bash[30576]: debug -271> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: got monmap 9 from mon.osd05 (according to old e9) Feb 04 09:32:14 osd07 bash[30576]: debug -270> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:14 osd07 bash[30576]: debug -269> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:14 osd07 bash[30576]: debug -268> 2021-02-04T08:32:14.025+0000 7f0101107700 10 monclient: dump: Feb 04 09:32:14 osd07 bash[30576]: epoch 9 Feb 04 09:32:14 osd07 bash[30576]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:14 osd07 bash[30576]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:14 osd07 bash[30576]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 15 (octopus) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:14 osd07 bash[30576]: debug -267> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:14 osd07 bash[30576]: debug -266> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:14 osd07 bash[30576]: debug -265> 2021-02-04T08:32:14.025+0000 7f00ff904700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:14 osd07 bash[30576]: debug -264> 2021-02-04T08:32:14.025+0000 7f0108390f40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:14 osd07 bash[30576]: debug -263> 2021-02-04T08:32:14.025+0000 7f0108390f40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:14 osd07 bash[30576]: debug -262> 2021-02-04T08:32:14.025+0000 7f0108390f40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:14 osd07 bash[30576]: debug -261> 2021-02-04T08:32:14.025+0000 7f0108390f40 5 asok(0x563972e0a000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:14 osd07 bash[30576]: debug -260> 2021-02-04T08:32:14.025+0000 7f0108390f40 5 asok(0x563972e0a000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:14 osd07 bash[30576]: debug -259> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command 0 hook 0x563972d62388 Feb 04 09:32:14 osd07 bash[30576]: debug -258> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command version hook 0x563972d62388 Feb 04 09:32:14 osd07 bash[30576]: debug -257> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command git_version hook 0x563972d62388 Feb 04 09:32:14 osd07 bash[30576]: debug -256> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command help hook 0x563972d5e370 Feb 04 09:32:14 osd07 bash[30576]: debug -255> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command get_command_descriptions hook 0x563972d5e070 Feb 04 09:32:14 osd07 bash[30576]: debug -254> 2021-02-04T08:32:14.029+0000 7f0100906700 5 asok(0x563972e0a000) entry start Feb 04 09:32:14 osd07 bash[30576]: debug -253> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 object store type is bluestore Feb 04 09:32:14 osd07 bash[30576]: debug -252> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:14 osd07 bash[30576]: debug -251> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -250> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -249> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -248> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -247> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -246> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -245> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -244> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -243> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -242> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -241> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -240> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -239> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:14 osd07 bash[30576]: debug -238> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -237> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -236> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -235> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -234> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -233> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -232> 2021-02-04T08:32:14.029+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -231> 2021-02-04T08:32:14.029+0000 7f00ff904700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -230> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -229> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug -228> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluestore bluefs available hook 0x563972d5e0b0 Feb 04 09:32:14 osd07 bash[30576]: debug -227> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluefs stats hook 0x563972d5e0b0 Feb 04 09:32:14 osd07 bash[30576]: debug -226> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -225> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -224> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -223> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -222> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -221> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -220> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -219> 2021-02-04T08:32:14.029+0000 7f00ff103700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -218> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -217> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:14 osd07 bash[30576]: debug -216> 2021-02-04T08:32:14.029+0000 7f00fe902700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -215> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) unregister_commands bluefs stats Feb 04 09:32:14 osd07 bash[30576]: debug -214> 2021-02-04T08:32:14.029+0000 7f0108390f40 5 asok(0x563972e0a000) unregister_commands bluestore bluefs available Feb 04 09:32:14 osd07 bash[30576]: debug -213> 2021-02-04T08:32:14.029+0000 7f0108390f40 20 bdev aio_wait 0x563972d92bb0 done Feb 04 09:32:14 osd07 bash[30576]: debug -212> 2021-02-04T08:32:14.029+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug -211> 2021-02-04T08:32:14.029+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug -210> 2021-02-04T08:32:14.281+0000 7f00ff103700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug -209> 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug -208> 2021-02-04T08:32:14.321+0000 7f00fe902700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug -207> 2021-02-04T08:32:14.321+0000 7f00fe902700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug -206> 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug -205> 2021-02-04T08:32:14.321+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug -204> 2021-02-04T08:32:14.321+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug -203> 2021-02-04T08:32:14.529+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug -202> 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug -201> 2021-02-04T08:32:14.561+0000 7f00ff904700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug -200> 2021-02-04T08:32:14.561+0000 7f00ff904700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug -199> 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug -198> 2021-02-04T08:32:14.561+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:14 osd07 bash[30576]: debug -197> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -196> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -195> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -194> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -193> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -192> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -191> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -190> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -189> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -188> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -187> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -186> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -185> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -184> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -183> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -182> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df5940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -181> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -180> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -179> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -178> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -177> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -176> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -175> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -174> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -173> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -172> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -171> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -170> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -169> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -168> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -167> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -166> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -165> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df6940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -164> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -163> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -162> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -161> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -160> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -159> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -158> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -157> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -156> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -155> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -154> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -153> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -152> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -151> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -150> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -149> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -148> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563972df7940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -147> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -146> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -145> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -144> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -143> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -142> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -141> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -140> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -139> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -138> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -137> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -136> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -135> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -134> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -133> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -132> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -131> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aae940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -130> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -129> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -128> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -127> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -126> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -125> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -124> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -123> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -122> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -121> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -120> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -119> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -118> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -117> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -116> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -115> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -114> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973aaf940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -113> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -112> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -111> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -110> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -109> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -108> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -107> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -106> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -105> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -104> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -103> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -102> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -101> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -100> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -99> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -98> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -97> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab0940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -96> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -95> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -94> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -93> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: cephx Feb 04 09:32:14 osd07 bash[30576]: debug -92> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding auth protocol: none Feb 04 09:32:14 osd07 bash[30576]: debug -91> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -90> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -89> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -88> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -87> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -86> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -85> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -84> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -83> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -82> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -81> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: crc Feb 04 09:32:14 osd07 bash[30576]: debug -80> 2021-02-04T08:32:14.561+0000 7f0108390f40 5 AuthRegistry(0x563973ab1940) adding con mode: secure Feb 04 09:32:14 osd07 bash[30576]: debug -79> 2021-02-04T08:32:14.561+0000 7f0108390f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:14 osd07 bash[30576]: debug -78> 2021-02-04T08:32:14.561+0000 7f0108390f40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:14 osd07 bash[30576]: debug -77> 2021-02-04T08:32:14.561+0000 7f0108390f40 -1 Falling back to public interface Feb 04 09:32:14 osd07 bash[30576]: debug -76> 2021-02-04T08:32:14.565+0000 7f0108390f40 10 monclient: build_initial_monmap Feb 04 09:32:14 osd07 bash[30576]: debug -75> 2021-02-04T08:32:14.565+0000 7f0108390f40 10 monclient: monmap: Feb 04 09:32:14 osd07 bash[30576]: epoch 0 Feb 04 09:32:14 osd07 bash[30576]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:14 osd07 bash[30576]: last_changed 0.000000 Feb 04 09:32:14 osd07 bash[30576]: created 0.000000 Feb 04 09:32:14 osd07 bash[30576]: min_mon_release 0 (unknown) Feb 04 09:32:14 osd07 bash[30576]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:14 osd07 bash[30576]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:14 osd07 bash[30576]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:14 osd07 bash[30576]: debug -74> 2021-02-04T08:32:14.577+0000 7f0108390f40 0 load: jerasure load: lrc load: isa Feb 04 09:32:14 osd07 bash[30576]: debug -73> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -72> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -71> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:14 osd07 bash[30576]: debug -70> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -69> 2021-02-04T08:32:14.577+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -68> 2021-02-04T08:32:14.577+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -67> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -66> 2021-02-04T08:32:14.577+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -65> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -64> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -63> 2021-02-04T08:32:14.577+0000 7f00fe902700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -62> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -61> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug -60> 2021-02-04T08:32:14.577+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:14 osd07 bash[30576]: debug -59> 2021-02-04T08:32:14.577+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:14 osd07 bash[30576]: debug -58> 2021-02-04T08:32:14.825+0000 7f0100105700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:14 osd07 bash[30576]: debug -57> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:14 osd07 bash[30576]: debug -56> 2021-02-04T08:32:14.869+0000 7f00fe902700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:14 osd07 bash[30576]: debug -55> 2021-02-04T08:32:14.869+0000 7f00fe902700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:14 osd07 bash[30576]: debug -54> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:14 osd07 bash[30576]: debug -53> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 asok(0x563972e0a000) register_command objecter_requests hook 0x563972d5e980 Feb 04 09:32:14 osd07 bash[30576]: debug -52> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -51> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -50> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -49> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -48> 2021-02-04T08:32:14.869+0000 7f0108390f40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:14 osd07 bash[30576]: debug -47> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -46> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -45> 2021-02-04T08:32:14.869+0000 7f0108390f40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:14 osd07 bash[30576]: debug -44> 2021-02-04T08:32:14.869+0000 7f0108390f40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:14 osd07 bash[30576]: debug -43> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:14 osd07 bash[30576]: debug -42> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:14 osd07 bash[30576]: debug -41> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -40> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -39> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -38> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -37> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:14 osd07 bash[30576]: debug -36> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -35> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -34> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -33> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -32> 2021-02-04T08:32:14.869+0000 7f00fb88b700 10 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -31> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -30> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -29> 2021-02-04T08:32:14.869+0000 7f00fb08a700 20 bdev(0x563973a9c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -28> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -27> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:14 osd07 bash[30576]: debug -26> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -25> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -24> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -23> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -22> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:14 osd07 bash[30576]: debug -21> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:14 osd07 bash[30576]: debug -20> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:14 osd07 bash[30576]: debug -19> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:14 osd07 bash[30576]: debug -18> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:14 osd07 bash[30576]: debug -17> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluestore bluefs available hook 0x563972d5e990 Feb 04 09:32:14 osd07 bash[30576]: debug -16> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 asok(0x563972e0a000) register_command bluefs stats hook 0x563972d5e990 Feb 04 09:32:14 osd07 bash[30576]: debug -15> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -14> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:14 osd07 bash[30576]: debug -13> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:14 osd07 bash[30576]: debug -12> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:14 osd07 bash[30576]: debug -11> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -10> 2021-02-04T08:32:14.869+0000 7f0108390f40 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:14 osd07 bash[30576]: debug -9> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:14 osd07 bash[30576]: debug -8> 2021-02-04T08:32:14.869+0000 7f00fa889700 10 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:14 osd07 bash[30576]: debug -7> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:14 osd07 bash[30576]: debug -6> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:14 osd07 bash[30576]: debug -5> 2021-02-04T08:32:14.869+0000 7f0108390f40 1 bluefs mount Feb 04 09:32:14 osd07 bash[30576]: debug -4> 2021-02-04T08:32:14.869+0000 7f0108390f40 10 bluefs _open_super Feb 04 09:32:14 osd07 bash[30576]: debug -3> 2021-02-04T08:32:14.869+0000 7f0108390f40 5 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:14 osd07 bash[30576]: debug -2> 2021-02-04T08:32:14.869+0000 7f00fa088700 20 bdev(0x563973a9c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:14 osd07 bash[30576]: debug -1> 2021-02-04T08:32:14.877+0000 7f0108390f40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f0108390f40 time 2021-02-04T08:32:14.872983+0000 Feb 04 09:32:14 osd07 bash[30576]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5639688b4372] Feb 04 09:32:14 osd07 bash[30576]: 2: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 4: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 5: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 10: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 11: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 12: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 13: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: debug 0> 2021-02-04T08:32:14.881+0000 7f0108390f40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:14 osd07 bash[30576]: in thread 7f0108390f40 thread_name:ceph-osd Feb 04 09:32:14 osd07 bash[30576]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:14 osd07 bash[30576]: 1: (()+0x12b20) [0x7f01060f8b20] Feb 04 09:32:14 osd07 bash[30576]: 2: (gsignal()+0x10f) [0x7f0104d607ff] Feb 04 09:32:14 osd07 bash[30576]: 3: (abort()+0x127) [0x7f0104d4ac35] Feb 04 09:32:14 osd07 bash[30576]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5639688b43c3] Feb 04 09:32:14 osd07 bash[30576]: 5: (()+0x50658c) [0x5639688b458c] Feb 04 09:32:14 osd07 bash[30576]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x563968f8cc3d] Feb 04 09:32:14 osd07 bash[30576]: 7: (BlueFS::_open_super()+0x155) [0x563968f2c0e5] Feb 04 09:32:14 osd07 bash[30576]: 8: (BlueFS::mount()+0xe9) [0x563968f50f99] Feb 04 09:32:14 osd07 bash[30576]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x563968e2e974] Feb 04 09:32:14 osd07 bash[30576]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x563968e2fe1c] Feb 04 09:32:14 osd07 bash[30576]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x563968e42d6c] Feb 04 09:32:14 osd07 bash[30576]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x563968e9b007] Feb 04 09:32:14 osd07 bash[30576]: 13: (OSD::init()+0x380) [0x5639689c2f20] Feb 04 09:32:14 osd07 bash[30576]: 14: (main()+0x47e4) [0x563968916264] Feb 04 09:32:14 osd07 bash[30576]: 15: (__libc_start_main()+0xf3) [0x7f0104d4c7b3] Feb 04 09:32:14 osd07 bash[30576]: 16: (_start()+0x2e) [0x56396894456e] Feb 04 09:32:14 osd07 bash[30576]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:14 osd07 bash[30576]: --- logging levels --- Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 none Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 lockdep Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 context Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 crush Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_balancer Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_locker Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_log Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_log_expire Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mds_migrator Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 buffer Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 timer Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 filer Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 striper Feb 04 09:32:14 osd07 bash[30576]: 0/ 1 objecter Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rados Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd_mirror Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd_replay Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 rbd_rwl Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 journaler Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 objectcacher Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 immutable_obj_cache Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 client Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 osd Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 optracker Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 objclass Feb 04 09:32:14 osd07 bash[30576]: 1/ 3 filestore Feb 04 09:32:14 osd07 bash[30576]: 1/ 3 journal Feb 04 09:32:14 osd07 bash[30576]: 0/ 0 ms Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mon Feb 04 09:32:14 osd07 bash[30576]: 0/10 monc Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 paxos Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 tp Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 auth Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 crypto Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 finisher Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 reserver Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 heartbeatmap Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 perfcounter Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 rgw Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 rgw_sync Feb 04 09:32:14 osd07 bash[30576]: 1/10 civetweb Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 javaclient Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 asok Feb 04 09:32:14 osd07 bash[30576]: 1/ 1 throttle Feb 04 09:32:14 osd07 bash[30576]: 0/ 0 refs Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 compressor Feb 04 09:32:14 osd07 bash[30576]: 20/20 bluestore Feb 04 09:32:14 osd07 bash[30576]: 20/20 bluefs Feb 04 09:32:14 osd07 bash[30576]: 20/20 bdev Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 kstore Feb 04 09:32:14 osd07 bash[30576]: 4/ 5 rocksdb Feb 04 09:32:14 osd07 bash[30576]: 4/ 5 leveldb Feb 04 09:32:14 osd07 bash[30576]: 4/ 5 memdb Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 fuse Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mgr Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 mgrc Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 dpdk Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 eventtrace Feb 04 09:32:14 osd07 bash[30576]: 1/ 5 prioritycache Feb 04 09:32:14 osd07 bash[30576]: 0/ 5 test Feb 04 09:32:14 osd07 bash[30576]: -2/-2 (syslog threshold) Feb 04 09:32:14 osd07 bash[30576]: 99/99 (stderr threshold) Feb 04 09:32:14 osd07 bash[30576]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:14 osd07 bash[30576]: 7f00fa088700 / bstore_discard Feb 04 09:32:14 osd07 bash[30576]: 7f00fa889700 / bstore_aio Feb 04 09:32:14 osd07 bash[30576]: 7f00fb08a700 / bstore_discard Feb 04 09:32:14 osd07 bash[30576]: 7f00fb88b700 / bstore_aio Feb 04 09:32:14 osd07 bash[30576]: 7f00fe902700 / OpHistorySvc Feb 04 09:32:14 osd07 bash[30576]: 7f00ff103700 / safe_timer Feb 04 09:32:14 osd07 bash[30576]: 7f00ff904700 / signal_handler Feb 04 09:32:14 osd07 bash[30576]: 7f0100105700 / ceph-osd Feb 04 09:32:14 osd07 bash[30576]: 7f0100906700 / admin_socket Feb 04 09:32:14 osd07 bash[30576]: 7f0101107700 / service Feb 04 09:32:14 osd07 bash[30576]: 7f0101908700 / msgr-worker-2 Feb 04 09:32:14 osd07 bash[30576]: 7f0102109700 / msgr-worker-1 Feb 04 09:32:14 osd07 bash[30576]: 7f010290a700 / msgr-worker-0 Feb 04 09:32:14 osd07 bash[30576]: 7f0108390f40 / ceph-osd Feb 04 09:32:14 osd07 bash[30576]: max_recent 10000 Feb 04 09:32:14 osd07 bash[30576]: max_new 1000 Feb 04 09:32:14 osd07 bash[30576]: log_file /var/lib/ceph/crash/2021-02-04T08:32:14.881824Z_f50dd446-e80f-4c34-ad92-a3162b750a5e/log Feb 04 09:32:14 osd07 bash[30576]: --- end dump of recent events --- Feb 04 09:32:14 osd07 bash[30576]: reraise_fatal: default handler for signal 6 didn't terminate the process? Feb 04 09:32:15 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Main process exited, code=exited, status=1/FAILURE Feb 04 09:32:15 osd07 bash[30794]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:15 osd07 bash[30794]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-deactivate Feb 04 09:32:15 osd07 bash[30794]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:16 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Failed with result 'exit-code'. Feb 04 09:32:26 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Service RestartSec=10s expired, scheduling restart. Feb 04 09:32:26 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Scheduled restart job, restart counter is at 1. Feb 04 09:32:26 osd07 systemd[1]: Stopped Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:26 osd07 systemd[1]: Starting Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e... Feb 04 09:32:26 osd07 docker[30973]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:26 osd07 docker[30973]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:26 osd07 systemd[1]: Started Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:26 osd07 bash[30988]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:26 osd07 bash[30988]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-activate Feb 04 09:32:26 osd07 bash[30988]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:27 osd07 bash[30988]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:27 osd07 bash[30988]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 --path /var/lib/ceph/osd/ceph-18 --no-mon-config Feb 04 09:32:27 osd07 bash[30988]: Running command: /usr/bin/ln -snf /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:27 osd07 bash[30988]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:27 osd07 bash[30988]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7 Feb 04 09:32:27 osd07 bash[30988]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:27 osd07 bash[30988]: --> ceph-volume lvm activate successful for osd ID: 18 Feb 04 09:32:27 osd07 bash[30988]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:27 osd07 bash[30988]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:27 osd07 bash[30988]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.765+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26ab8c5700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26ab0c4700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26aa8c3700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev aio_wait 0x55cd2215f3f0 done Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:27 osd07 bash[30988]: debug 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.017+0000 7f26ab0c4700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.049+0000 7f26aa8c3700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.049+0000 7f26aa8c3700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.049+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.269+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26ab8c5700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26ab8c5700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26b4351f40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.301+0000 7f26b4351f40 -1 Falling back to public interface Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 0 load: jerasure load: lrc load: isa Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26aa8c3700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.565+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26aa8c3700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26aa8c3700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26a784c700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26a704b700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f26b4351f40 time 2021-02-04T08:32:28.609047+0000 Feb 04 09:32:28 osd07 bash[30988]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26a6049700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluefs mount Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluefs _open_super Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x55cd16fa4372] Feb 04 09:32:28 osd07 bash[30988]: 2: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 4: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 5: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 10: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 11: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 12: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 13: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: *** Caught signal (Aborted) ** Feb 04 09:32:28 osd07 bash[30988]: in thread 7f26b4351f40 thread_name:ceph-osd Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.605+0000 7f26a684a700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.609+0000 7f26b4351f40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f26b4351f40 time 2021-02-04T08:32:28.609047+0000 Feb 04 09:32:28 osd07 bash[30988]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x55cd16fa4372] Feb 04 09:32:28 osd07 bash[30988]: 2: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 4: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 5: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 10: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 11: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 12: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 13: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (()+0x12b20) [0x7f26b20b9b20] Feb 04 09:32:28 osd07 bash[30988]: 2: (gsignal()+0x10f) [0x7f26b0d217ff] Feb 04 09:32:28 osd07 bash[30988]: 3: (abort()+0x127) [0x7f26b0d0bc35] Feb 04 09:32:28 osd07 bash[30988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x55cd16fa43c3] Feb 04 09:32:28 osd07 bash[30988]: 5: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 13: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 14: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 15: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 16: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: debug 2021-02-04T08:32:28.613+0000 7f26b4351f40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:28 osd07 bash[30988]: in thread 7f26b4351f40 thread_name:ceph-osd Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (()+0x12b20) [0x7f26b20b9b20] Feb 04 09:32:28 osd07 bash[30988]: 2: (gsignal()+0x10f) [0x7f26b0d217ff] Feb 04 09:32:28 osd07 bash[30988]: 3: (abort()+0x127) [0x7f26b0d0bc35] Feb 04 09:32:28 osd07 bash[30988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x55cd16fa43c3] Feb 04 09:32:28 osd07 bash[30988]: 5: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 13: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 14: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 15: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 16: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:28 osd07 bash[30988]: --- begin dump of recent events --- Feb 04 09:32:28 osd07 bash[30988]: debug -376> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command assert hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -375> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command abort hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -374> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command leak_some_memory hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -373> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perfcounters_dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -372> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command 1 hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -371> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -370> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perfcounters_schema hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -369> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf histogram dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -368> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command 2 hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -367> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf schema hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -366> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf histogram schema hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -365> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf reset hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -364> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config show hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -363> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config help hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -362> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config set hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -361> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config unset hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -360> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config get hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -359> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config diff hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -358> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config diff get hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -357> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command injectargs hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -356> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command log flush hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -355> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command log dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -354> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command log reopen hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -353> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command dump_mempools hook 0x55cd22daa068 Feb 04 09:32:28 osd07 bash[30988]: debug -352> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: get_monmap_and_config Feb 04 09:32:28 osd07 bash[30988]: debug -351> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: build_initial_monmap Feb 04 09:32:28 osd07 bash[30988]: debug -350> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: monmap: Feb 04 09:32:28 osd07 bash[30988]: epoch 0 Feb 04 09:32:28 osd07 bash[30988]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:28 osd07 bash[30988]: last_changed 2021-02-04T08:32:27.762822+0000 Feb 04 09:32:28 osd07 bash[30988]: created 2021-02-04T08:32:27.762822+0000 Feb 04 09:32:28 osd07 bash[30988]: min_mon_release 0 (unknown) Feb 04 09:32:28 osd07 bash[30988]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:28 osd07 bash[30988]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:28 osd07 bash[30988]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:28 osd07 bash[30988]: debug -349> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -348> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -347> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -346> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -345> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -344> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -343> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -342> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -341> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -340> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -339> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -338> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -337> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -336> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -335> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -334> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -333> 2021-02-04T08:32:27.761+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -332> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: init Feb 04 09:32:28 osd07 bash[30988]: debug -331> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -330> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -329> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -328> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -327> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -326> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -325> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -324> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -323> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -322> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -321> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -320> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -319> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -318> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -317> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -316> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -315> 2021-02-04T08:32:27.761+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -314> 2021-02-04T08:32:27.761+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -313> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: _reopen_session rank -1 Feb 04 09:32:28 osd07 bash[30988]: debug -312> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: _add_conns ranks=[0,1,2] Feb 04 09:32:28 osd07 bash[30988]: debug -311> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): picked mon.noname-a con 0x55cd221d4c00 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:28 osd07 bash[30988]: debug -310> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): picked mon.noname-b con 0x55cd221d5000 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:28 osd07 bash[30988]: debug -309> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): picked mon.noname-c con 0x55cd221d5400 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:28 osd07 bash[30988]: debug -308> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:28 osd07 bash[30988]: debug -307> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:28 osd07 bash[30988]: debug -306> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:28 osd07 bash[30988]: debug -305> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): _renew_subs Feb 04 09:32:28 osd07 bash[30988]: debug -304> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:27.764063+0000 Feb 04 09:32:28 osd07 bash[30988]: debug -303> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): get_auth_request con 0x55cd221d4c00 auth_method 0 Feb 04 09:32:28 osd07 bash[30988]: debug -302> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:28 osd07 bash[30988]: debug -301> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:28 osd07 bash[30988]: debug -300> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): get_auth_request con 0x55cd221d5400 auth_method 0 Feb 04 09:32:28 osd07 bash[30988]: debug -299> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:28 osd07 bash[30988]: debug -298> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:28 osd07 bash[30988]: debug -297> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): get_auth_request con 0x55cd221d5000 auth_method 0 Feb 04 09:32:28 osd07 bash[30988]: debug -296> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:28 osd07 bash[30988]: debug -295> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:28 osd07 bash[30988]: debug -294> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:28 osd07 bash[30988]: debug -293> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:28 osd07 bash[30988]: debug -292> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:28 osd07 bash[30988]: debug -291> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:28 osd07 bash[30988]: debug -290> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:28 osd07 bash[30988]: debug -289> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:28 osd07 bash[30988]: debug -288> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:28 osd07 bash[30988]: debug -287> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:28 osd07 bash[30988]: debug -286> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:28 osd07 bash[30988]: debug -285> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_done global_id 3626479 payload 450 Feb 04 09:32:28 osd07 bash[30988]: debug -284> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient: _finish_hunting 0 Feb 04 09:32:28 osd07 bash[30988]: debug -283> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 1 monclient: found mon.noname-c Feb 04 09:32:28 osd07 bash[30988]: debug -282> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient: _send_mon_message to mon.noname-c at v2:10.10.0.16:3300/0 Feb 04 09:32:28 osd07 bash[30988]: debug -281> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:28 osd07 bash[30988]: debug -280> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: got monmap 9 from mon.noname-c (according to old e9) Feb 04 09:32:28 osd07 bash[30988]: debug -279> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: dump: Feb 04 09:32:28 osd07 bash[30988]: epoch 9 Feb 04 09:32:28 osd07 bash[30988]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:28 osd07 bash[30988]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:28 osd07 bash[30988]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:28 osd07 bash[30988]: min_mon_release 15 (octopus) Feb 04 09:32:28 osd07 bash[30988]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:28 osd07 bash[30988]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:28 osd07 bash[30988]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:28 osd07 bash[30988]: debug -278> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: _finish_auth 0 Feb 04 09:32:28 osd07 bash[30988]: debug -277> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:31:57.766199+0000) Feb 04 09:32:28 osd07 bash[30988]: debug -276> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: _send_mon_message to mon.osd06 at v2:10.10.0.16:3300/0 Feb 04 09:32:28 osd07 bash[30988]: debug -275> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 monclient: authenticate success, global_id 3626479 Feb 04 09:32:28 osd07 bash[30988]: debug -274> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:28 osd07 bash[30988]: debug -273> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 monclient: get_monmap_and_config success Feb 04 09:32:28 osd07 bash[30988]: debug -272> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 monclient: shutdown Feb 04 09:32:28 osd07 bash[30988]: debug -271> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 4 set_mon_vals no callback set Feb 04 09:32:28 osd07 bash[30988]: debug -270> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 Feb 04 09:32:28 osd07 bash[30988]: debug -269> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:28 osd07 bash[30988]: debug -268> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:28 osd07 bash[30988]: debug -267> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:28 osd07 bash[30988]: debug -266> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:28 osd07 bash[30988]: debug -265> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:28 osd07 bash[30988]: debug -264> 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:28 osd07 bash[30988]: debug -263> 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:28 osd07 bash[30988]: debug -262> 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:28 osd07 bash[30988]: debug -261> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:28 osd07 bash[30988]: debug -260> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:28 osd07 bash[30988]: debug -259> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command 0 hook 0x55cd2212e388 Feb 04 09:32:28 osd07 bash[30988]: debug -258> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command version hook 0x55cd2212e388 Feb 04 09:32:28 osd07 bash[30988]: debug -257> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command git_version hook 0x55cd2212e388 Feb 04 09:32:28 osd07 bash[30988]: debug -256> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command help hook 0x55cd2212a4b0 Feb 04 09:32:28 osd07 bash[30988]: debug -255> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command get_command_descriptions hook 0x55cd2212a070 Feb 04 09:32:28 osd07 bash[30988]: debug -254> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 object store type is bluestore Feb 04 09:32:28 osd07 bash[30988]: debug -253> 2021-02-04T08:32:27.765+0000 7f26ac8c7700 5 asok(0x55cd221d6000) entry start Feb 04 09:32:28 osd07 bash[30988]: debug -252> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:28 osd07 bash[30988]: debug -251> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -250> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -249> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -248> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -247> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -246> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -245> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -244> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -243> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -242> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -241> 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -240> 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -239> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:28 osd07 bash[30988]: debug -238> 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -237> 2021-02-04T08:32:27.765+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -236> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -235> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -234> 2021-02-04T08:32:27.769+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -233> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -232> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -231> 2021-02-04T08:32:27.769+0000 7f26ab8c5700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -230> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -229> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug -228> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluestore bluefs available hook 0x55cd2212a0b0 Feb 04 09:32:28 osd07 bash[30988]: debug -227> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluefs stats hook 0x55cd2212a0b0 Feb 04 09:32:28 osd07 bash[30988]: debug -226> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -225> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -224> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -223> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -222> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -221> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -220> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -219> 2021-02-04T08:32:27.769+0000 7f26ab0c4700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -218> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -217> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:28 osd07 bash[30988]: debug -216> 2021-02-04T08:32:27.769+0000 7f26aa8c3700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -215> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) unregister_commands bluefs stats Feb 04 09:32:28 osd07 bash[30988]: debug -214> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) unregister_commands bluestore bluefs available Feb 04 09:32:28 osd07 bash[30988]: debug -213> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev aio_wait 0x55cd2215f3f0 done Feb 04 09:32:28 osd07 bash[30988]: debug -212> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug -211> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug -210> 2021-02-04T08:32:28.017+0000 7f26ab0c4700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug -209> 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug -208> 2021-02-04T08:32:28.049+0000 7f26aa8c3700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug -207> 2021-02-04T08:32:28.049+0000 7f26aa8c3700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug -206> 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug -205> 2021-02-04T08:32:28.049+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug -204> 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug -203> 2021-02-04T08:32:28.269+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug -202> 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug -201> 2021-02-04T08:32:28.301+0000 7f26ab8c5700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug -200> 2021-02-04T08:32:28.301+0000 7f26ab8c5700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug -199> 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug -198> 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:28 osd07 bash[30988]: debug -197> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -196> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -195> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -194> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -193> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -192> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -191> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -190> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -189> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -188> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -187> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -186> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -185> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -184> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -183> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -182> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -181> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -180> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -179> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -178> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -177> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -176> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -175> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -174> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -173> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -172> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -171> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -170> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -169> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -168> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -167> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -166> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -165> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -164> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -163> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -162> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -161> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -160> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -159> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -158> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -157> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -156> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -155> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -154> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -153> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -152> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -151> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -150> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -149> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -148> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -147> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -146> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -145> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -144> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -143> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -142> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -141> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -140> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -139> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -138> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -137> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -136> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -135> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -134> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -133> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -132> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -131> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -130> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -129> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -128> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -127> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -126> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -125> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -124> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -123> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -122> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -121> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -120> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -119> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -118> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -117> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -116> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -115> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -114> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -113> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -112> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -111> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -110> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -109> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -108> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -107> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -106> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -105> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -104> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -103> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -102> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -101> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -100> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -99> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -98> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -97> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -96> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -95> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -94> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -93> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -92> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -91> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -90> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -89> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -88> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -87> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -86> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -85> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -84> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -83> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -82> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -81> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -80> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -79> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -78> 2021-02-04T08:32:28.301+0000 7f26b4351f40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:28 osd07 bash[30988]: debug -77> 2021-02-04T08:32:28.301+0000 7f26b4351f40 -1 Falling back to public interface Feb 04 09:32:28 osd07 bash[30988]: debug -76> 2021-02-04T08:32:28.305+0000 7f26b4351f40 10 monclient: build_initial_monmap Feb 04 09:32:28 osd07 bash[30988]: debug -75> 2021-02-04T08:32:28.305+0000 7f26b4351f40 10 monclient: monmap: Feb 04 09:32:28 osd07 bash[30988]: epoch 0 Feb 04 09:32:28 osd07 bash[30988]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:28 osd07 bash[30988]: last_changed 0.000000 Feb 04 09:32:28 osd07 bash[30988]: created 0.000000 Feb 04 09:32:28 osd07 bash[30988]: min_mon_release 0 (unknown) Feb 04 09:32:28 osd07 bash[30988]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:28 osd07 bash[30988]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:28 osd07 bash[30988]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:28 osd07 bash[30988]: debug -74> 2021-02-04T08:32:28.313+0000 7f26b4351f40 0 load: jerasure load: lrc load: isa Feb 04 09:32:28 osd07 bash[30988]: debug -73> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -72> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -71> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:28 osd07 bash[30988]: debug -70> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -69> 2021-02-04T08:32:28.313+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -68> 2021-02-04T08:32:28.313+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -67> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -66> 2021-02-04T08:32:28.313+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -65> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -64> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -63> 2021-02-04T08:32:28.313+0000 7f26aa8c3700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -62> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -61> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug -60> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug -59> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug -58> 2021-02-04T08:32:28.565+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug -57> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug -56> 2021-02-04T08:32:28.605+0000 7f26aa8c3700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug -55> 2021-02-04T08:32:28.605+0000 7f26aa8c3700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug -54> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug -53> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command objecter_requests hook 0x55cd2212a970 Feb 04 09:32:28 osd07 bash[30988]: debug -52> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -51> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -50> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -49> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -48> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -47> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -46> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -45> 2021-02-04T08:32:28.605+0000 7f26b4351f40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:28 osd07 bash[30988]: debug -44> 2021-02-04T08:32:28.605+0000 7f26b4351f40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:28 osd07 bash[30988]: debug -43> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:28 osd07 bash[30988]: debug -42> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:28 osd07 bash[30988]: debug -41> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -40> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -39> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -38> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -37> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:28 osd07 bash[30988]: debug -36> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -35> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -34> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -33> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -32> 2021-02-04T08:32:28.605+0000 7f26a784c700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -31> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -30> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -29> 2021-02-04T08:32:28.605+0000 7f26a704b700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -28> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -27> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug -26> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -25> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -24> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -23> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -22> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:28 osd07 bash[30988]: debug -21> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -20> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -19> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:28 osd07 bash[30988]: debug -18> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:28 osd07 bash[30988]: debug -17> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluestore bluefs available hook 0x55cd2212a980 Feb 04 09:32:28 osd07 bash[30988]: debug -16> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluefs stats hook 0x55cd2212a980 Feb 04 09:32:28 osd07 bash[30988]: debug -15> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -14> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -13> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -12> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -11> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -10> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -9> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -8> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -7> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:28 osd07 bash[30988]: debug -6> 2021-02-04T08:32:28.605+0000 7f26a6049700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -5> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluefs mount Feb 04 09:32:28 osd07 bash[30988]: debug -4> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluefs _open_super Feb 04 09:32:28 osd07 bash[30988]: debug -3> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:28 osd07 bash[30988]: debug -2> 2021-02-04T08:32:28.605+0000 7f26a684a700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -1> 2021-02-04T08:32:28.609+0000 7f26b4351f40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f26b4351f40 time 2021-02-04T08:32:28.609047+0000 Feb 04 09:32:28 osd07 bash[30988]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x55cd16fa4372] Feb 04 09:32:28 osd07 bash[30988]: 2: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 4: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 5: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 10: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 11: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 12: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 13: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: debug 0> 2021-02-04T08:32:28.613+0000 7f26b4351f40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:28 osd07 bash[30988]: in thread 7f26b4351f40 thread_name:ceph-osd Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (()+0x12b20) [0x7f26b20b9b20] Feb 04 09:32:28 osd07 bash[30988]: 2: (gsignal()+0x10f) [0x7f26b0d217ff] Feb 04 09:32:28 osd07 bash[30988]: 3: (abort()+0x127) [0x7f26b0d0bc35] Feb 04 09:32:28 osd07 bash[30988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x55cd16fa43c3] Feb 04 09:32:28 osd07 bash[30988]: 5: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 13: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 14: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 15: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 16: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:28 osd07 bash[30988]: --- logging levels --- Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 none Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 lockdep Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 context Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 crush Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_balancer Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_locker Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_log Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_log_expire Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_migrator Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 buffer Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 timer Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 filer Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 striper Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 objecter Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rados Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd_mirror Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd_replay Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd_rwl Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 journaler Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 objectcacher Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 immutable_obj_cache Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 client Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 osd Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 optracker Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 objclass Feb 04 09:32:28 osd07 bash[30988]: 1/ 3 filestore Feb 04 09:32:28 osd07 bash[30988]: 1/ 3 journal Feb 04 09:32:28 osd07 bash[30988]: 0/ 0 ms Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mon Feb 04 09:32:28 osd07 bash[30988]: 0/10 monc Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 paxos Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 tp Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 auth Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 crypto Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 finisher Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 reserver Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 heartbeatmap Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 perfcounter Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 rgw Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 rgw_sync Feb 04 09:32:28 osd07 bash[30988]: 1/10 civetweb Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 javaclient Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 asok Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 throttle Feb 04 09:32:28 osd07 bash[30988]: 0/ 0 refs Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 compressor Feb 04 09:32:28 osd07 bash[30988]: 20/20 bluestore Feb 04 09:32:28 osd07 bash[30988]: 20/20 bluefs Feb 04 09:32:28 osd07 bash[30988]: 20/20 bdev Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 kstore Feb 04 09:32:28 osd07 bash[30988]: 4/ 5 rocksdb Feb 04 09:32:28 osd07 bash[30988]: 4/ 5 leveldb Feb 04 09:32:28 osd07 bash[30988]: 4/ 5 memdb Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 fuse Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mgr Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mgrc Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 dpdk Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 eventtrace Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 prioritycache Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 test Feb 04 09:32:28 osd07 bash[30988]: -2/-2 (syslog threshold) Feb 04 09:32:28 osd07 bash[30988]: 99/99 (stderr threshold) Feb 04 09:32:28 osd07 bash[30988]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:28 osd07 bash[30988]: 7f26a6049700 / bstore_discard Feb 04 09:32:28 osd07 bash[30988]: 7f26a684a700 / bstore_aio Feb 04 09:32:28 osd07 bash[30988]: 7f26a704b700 / bstore_discard Feb 04 09:32:28 osd07 bash[30988]: 7f26a784c700 / bstore_aio Feb 04 09:32:28 osd07 bash[30988]: 7f26aa8c3700 / OpHistorySvc Feb 04 09:32:28 osd07 bash[30988]: 7f26ab0c4700 / safe_timer Feb 04 09:32:28 osd07 bash[30988]: 7f26ab8c5700 / signal_handler Feb 04 09:32:28 osd07 bash[30988]: 7f26ac0c6700 / ceph-osd Feb 04 09:32:28 osd07 bash[30988]: 7f26ac8c7700 / admin_socket Feb 04 09:32:28 osd07 bash[30988]: 7f26ad0c8700 / service Feb 04 09:32:28 osd07 bash[30988]: 7f26ad8c9700 / msgr-worker-2 Feb 04 09:32:28 osd07 bash[30988]: 7f26ae0ca700 / msgr-worker-1 Feb 04 09:32:28 osd07 bash[30988]: 7f26ae8cb700 / msgr-worker-0 Feb 04 09:32:28 osd07 bash[30988]: 7f26b4351f40 / ceph-osd Feb 04 09:32:28 osd07 bash[30988]: max_recent 10000 Feb 04 09:32:28 osd07 bash[30988]: max_new 1000 Feb 04 09:32:28 osd07 bash[30988]: log_file Feb 04 09:32:28 osd07 bash[30988]: --- end dump of recent events --- Feb 04 09:32:28 osd07 bash[30988]: --- begin dump of recent events --- Feb 04 09:32:28 osd07 bash[30988]: debug -376> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command assert hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -375> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command abort hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -374> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command leak_some_memory hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -373> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perfcounters_dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -372> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command 1 hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -371> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -370> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perfcounters_schema hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -369> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf histogram dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -368> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command 2 hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -367> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf schema hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -366> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf histogram schema hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -365> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command perf reset hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -364> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config show hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -363> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config help hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -362> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config set hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -361> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config unset hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -360> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config get hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -359> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config diff hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -358> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command config diff get hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -357> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command injectargs hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -356> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command log flush hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -355> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command log dump hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -354> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command log reopen hook 0x55cd2212a5b0 Feb 04 09:32:28 osd07 bash[30988]: debug -353> 2021-02-04T08:32:27.753+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command dump_mempools hook 0x55cd22daa068 Feb 04 09:32:28 osd07 bash[30988]: debug -352> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: get_monmap_and_config Feb 04 09:32:28 osd07 bash[30988]: debug -351> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: build_initial_monmap Feb 04 09:32:28 osd07 bash[30988]: debug -350> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: monmap: Feb 04 09:32:28 osd07 bash[30988]: epoch 0 Feb 04 09:32:28 osd07 bash[30988]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:28 osd07 bash[30988]: last_changed 2021-02-04T08:32:27.762822+0000 Feb 04 09:32:28 osd07 bash[30988]: created 2021-02-04T08:32:27.762822+0000 Feb 04 09:32:28 osd07 bash[30988]: min_mon_release 0 (unknown) Feb 04 09:32:28 osd07 bash[30988]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:28 osd07 bash[30988]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:28 osd07 bash[30988]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:28 osd07 bash[30988]: debug -349> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -348> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -347> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -346> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -345> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -344> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -343> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -342> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -341> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -340> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -339> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -338> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -337> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -336> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -335> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -334> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c0940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -333> 2021-02-04T08:32:27.761+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -332> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: init Feb 04 09:32:28 osd07 bash[30988]: debug -331> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -330> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -329> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -328> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -327> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -326> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -325> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -324> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -323> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -322> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -321> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -320> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -319> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -318> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -317> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -316> 2021-02-04T08:32:27.761+0000 7f26b4351f40 5 AuthRegistry(0x7ffea3121710) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -315> 2021-02-04T08:32:27.761+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -314> 2021-02-04T08:32:27.761+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -313> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: _reopen_session rank -1 Feb 04 09:32:28 osd07 bash[30988]: debug -312> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient: _add_conns ranks=[0,1,2] Feb 04 09:32:28 osd07 bash[30988]: debug -311> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): picked mon.noname-a con 0x55cd221d4c00 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:28 osd07 bash[30988]: debug -310> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): picked mon.noname-b con 0x55cd221d5000 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:28 osd07 bash[30988]: debug -309> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): picked mon.noname-c con 0x55cd221d5400 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:28 osd07 bash[30988]: debug -308> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:28 osd07 bash[30988]: debug -307> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:28 osd07 bash[30988]: debug -306> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): start opening mon connection Feb 04 09:32:28 osd07 bash[30988]: debug -305> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): _renew_subs Feb 04 09:32:28 osd07 bash[30988]: debug -304> 2021-02-04T08:32:27.761+0000 7f26b4351f40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:27.764063+0000 Feb 04 09:32:28 osd07 bash[30988]: debug -303> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): get_auth_request con 0x55cd221d4c00 auth_method 0 Feb 04 09:32:28 osd07 bash[30988]: debug -302> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:28 osd07 bash[30988]: debug -301> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:28 osd07 bash[30988]: debug -300> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): get_auth_request con 0x55cd221d5400 auth_method 0 Feb 04 09:32:28 osd07 bash[30988]: debug -299> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:28 osd07 bash[30988]: debug -298> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:28 osd07 bash[30988]: debug -297> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): get_auth_request con 0x55cd221d5000 auth_method 0 Feb 04 09:32:28 osd07 bash[30988]: debug -296> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:28 osd07 bash[30988]: debug -295> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:28 osd07 bash[30988]: debug -294> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:28 osd07 bash[30988]: debug -293> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:28 osd07 bash[30988]: debug -292> 2021-02-04T08:32:27.761+0000 7f26ae0ca700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:28 osd07 bash[30988]: debug -291> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:28 osd07 bash[30988]: debug -290> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:28 osd07 bash[30988]: debug -289> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:28 osd07 bash[30988]: debug -288> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:28 osd07 bash[30988]: debug -287> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:28 osd07 bash[30988]: debug -286> 2021-02-04T08:32:27.761+0000 7f26ad8c9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:28 osd07 bash[30988]: debug -285> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient(hunting): handle_auth_done global_id 3626479 payload 450 Feb 04 09:32:28 osd07 bash[30988]: debug -284> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient: _finish_hunting 0 Feb 04 09:32:28 osd07 bash[30988]: debug -283> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 1 monclient: found mon.noname-c Feb 04 09:32:28 osd07 bash[30988]: debug -282> 2021-02-04T08:32:27.761+0000 7f26ae8cb700 10 monclient: _send_mon_message to mon.noname-c at v2:10.10.0.16:3300/0 Feb 04 09:32:28 osd07 bash[30988]: debug -281> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:28 osd07 bash[30988]: debug -280> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: got monmap 9 from mon.noname-c (according to old e9) Feb 04 09:32:28 osd07 bash[30988]: debug -279> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: dump: Feb 04 09:32:28 osd07 bash[30988]: epoch 9 Feb 04 09:32:28 osd07 bash[30988]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:28 osd07 bash[30988]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:28 osd07 bash[30988]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:28 osd07 bash[30988]: min_mon_release 15 (octopus) Feb 04 09:32:28 osd07 bash[30988]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:28 osd07 bash[30988]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:28 osd07 bash[30988]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:28 osd07 bash[30988]: debug -278> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: _finish_auth 0 Feb 04 09:32:28 osd07 bash[30988]: debug -277> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:31:57.766199+0000) Feb 04 09:32:28 osd07 bash[30988]: debug -276> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: _send_mon_message to mon.osd06 at v2:10.10.0.16:3300/0 Feb 04 09:32:28 osd07 bash[30988]: debug -275> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 monclient: authenticate success, global_id 3626479 Feb 04 09:32:28 osd07 bash[30988]: debug -274> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:28 osd07 bash[30988]: debug -273> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 monclient: get_monmap_and_config success Feb 04 09:32:28 osd07 bash[30988]: debug -272> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 monclient: shutdown Feb 04 09:32:28 osd07 bash[30988]: debug -271> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 4 set_mon_vals no callback set Feb 04 09:32:28 osd07 bash[30988]: debug -270> 2021-02-04T08:32:27.765+0000 7f26ad0c8700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 Feb 04 09:32:28 osd07 bash[30988]: debug -269> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:28 osd07 bash[30988]: debug -268> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:28 osd07 bash[30988]: debug -267> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:28 osd07 bash[30988]: debug -266> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:28 osd07 bash[30988]: debug -265> 2021-02-04T08:32:27.765+0000 7f26ab8c5700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:28 osd07 bash[30988]: debug -264> 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:28 osd07 bash[30988]: debug -263> 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:28 osd07 bash[30988]: debug -262> 2021-02-04T08:32:27.765+0000 7f26b4351f40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:28 osd07 bash[30988]: debug -261> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:28 osd07 bash[30988]: debug -260> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:28 osd07 bash[30988]: debug -259> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command 0 hook 0x55cd2212e388 Feb 04 09:32:28 osd07 bash[30988]: debug -258> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command version hook 0x55cd2212e388 Feb 04 09:32:28 osd07 bash[30988]: debug -257> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command git_version hook 0x55cd2212e388 Feb 04 09:32:28 osd07 bash[30988]: debug -256> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command help hook 0x55cd2212a4b0 Feb 04 09:32:28 osd07 bash[30988]: debug -255> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command get_command_descriptions hook 0x55cd2212a070 Feb 04 09:32:28 osd07 bash[30988]: debug -254> 2021-02-04T08:32:27.765+0000 7f26b4351f40 5 object store type is bluestore Feb 04 09:32:28 osd07 bash[30988]: debug -253> 2021-02-04T08:32:27.765+0000 7f26ac8c7700 5 asok(0x55cd221d6000) entry start Feb 04 09:32:28 osd07 bash[30988]: debug -252> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:28 osd07 bash[30988]: debug -251> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -250> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -249> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -248> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -247> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -246> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -245> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -244> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -243> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -242> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -241> 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -240> 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -239> 2021-02-04T08:32:27.765+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:28 osd07 bash[30988]: debug -238> 2021-02-04T08:32:27.765+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -237> 2021-02-04T08:32:27.765+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -236> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -235> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -234> 2021-02-04T08:32:27.769+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -233> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -232> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -231> 2021-02-04T08:32:27.769+0000 7f26ab8c5700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -230> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -229> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug -228> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluestore bluefs available hook 0x55cd2212a0b0 Feb 04 09:32:28 osd07 bash[30988]: debug -227> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluefs stats hook 0x55cd2212a0b0 Feb 04 09:32:28 osd07 bash[30988]: debug -226> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -225> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -224> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -223> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -222> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -221> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -220> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -219> 2021-02-04T08:32:27.769+0000 7f26ab0c4700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -218> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -217> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:28 osd07 bash[30988]: debug -216> 2021-02-04T08:32:27.769+0000 7f26aa8c3700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -215> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) unregister_commands bluefs stats Feb 04 09:32:28 osd07 bash[30988]: debug -214> 2021-02-04T08:32:27.769+0000 7f26b4351f40 5 asok(0x55cd221d6000) unregister_commands bluestore bluefs available Feb 04 09:32:28 osd07 bash[30988]: debug -213> 2021-02-04T08:32:27.769+0000 7f26b4351f40 20 bdev aio_wait 0x55cd2215f3f0 done Feb 04 09:32:28 osd07 bash[30988]: debug -212> 2021-02-04T08:32:27.769+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug -211> 2021-02-04T08:32:27.769+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug -210> 2021-02-04T08:32:28.017+0000 7f26ab0c4700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug -209> 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug -208> 2021-02-04T08:32:28.049+0000 7f26aa8c3700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug -207> 2021-02-04T08:32:28.049+0000 7f26aa8c3700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug -206> 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug -205> 2021-02-04T08:32:28.049+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug -204> 2021-02-04T08:32:28.049+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug -203> 2021-02-04T08:32:28.269+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug -202> 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug -201> 2021-02-04T08:32:28.301+0000 7f26ab8c5700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug -200> 2021-02-04T08:32:28.301+0000 7f26ab8c5700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug -199> 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug -198> 2021-02-04T08:32:28.301+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:28 osd07 bash[30988]: debug -197> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -196> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -195> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -194> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -193> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -192> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -191> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -190> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -189> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -188> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -187> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -186> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -185> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -184> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -183> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -182> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c1940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -181> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -180> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -179> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -178> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -177> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -176> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -175> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -174> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -173> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -172> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -171> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -170> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -169> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -168> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -167> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -166> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -165> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c2940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -164> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -163> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -162> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -161> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -160> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -159> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -158> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -157> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -156> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -155> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -154> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -153> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -152> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -151> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -150> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -149> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -148> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd221c3940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -147> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -146> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -145> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -144> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -143> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -142> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -141> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -140> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -139> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -138> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -137> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -136> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -135> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -134> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -133> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -132> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -131> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e42940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -130> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -129> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -128> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -127> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -126> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -125> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -124> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -123> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -122> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -121> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -120> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -119> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -118> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -117> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -116> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -115> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -114> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e43940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -113> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -112> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -111> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -110> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -109> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -108> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -107> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -106> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -105> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -104> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -103> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -102> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -101> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -100> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -99> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -98> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -97> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e44940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -96> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -95> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -94> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -93> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: cephx Feb 04 09:32:28 osd07 bash[30988]: debug -92> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding auth protocol: none Feb 04 09:32:28 osd07 bash[30988]: debug -91> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -90> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -89> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -88> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -87> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -86> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -85> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -84> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -83> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -82> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -81> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: crc Feb 04 09:32:28 osd07 bash[30988]: debug -80> 2021-02-04T08:32:28.301+0000 7f26b4351f40 5 AuthRegistry(0x55cd22e45940) adding con mode: secure Feb 04 09:32:28 osd07 bash[30988]: debug -79> 2021-02-04T08:32:28.301+0000 7f26b4351f40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:28 osd07 bash[30988]: debug -78> 2021-02-04T08:32:28.301+0000 7f26b4351f40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:28 osd07 bash[30988]: debug -77> 2021-02-04T08:32:28.301+0000 7f26b4351f40 -1 Falling back to public interface Feb 04 09:32:28 osd07 bash[30988]: debug -76> 2021-02-04T08:32:28.305+0000 7f26b4351f40 10 monclient: build_initial_monmap Feb 04 09:32:28 osd07 bash[30988]: debug -75> 2021-02-04T08:32:28.305+0000 7f26b4351f40 10 monclient: monmap: Feb 04 09:32:28 osd07 bash[30988]: epoch 0 Feb 04 09:32:28 osd07 bash[30988]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:28 osd07 bash[30988]: last_changed 0.000000 Feb 04 09:32:28 osd07 bash[30988]: created 0.000000 Feb 04 09:32:28 osd07 bash[30988]: min_mon_release 0 (unknown) Feb 04 09:32:28 osd07 bash[30988]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:28 osd07 bash[30988]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:28 osd07 bash[30988]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:28 osd07 bash[30988]: debug -74> 2021-02-04T08:32:28.313+0000 7f26b4351f40 0 load: jerasure load: lrc load: isa Feb 04 09:32:28 osd07 bash[30988]: debug -73> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -72> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -71> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:28 osd07 bash[30988]: debug -70> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -69> 2021-02-04T08:32:28.313+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -68> 2021-02-04T08:32:28.313+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -67> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -66> 2021-02-04T08:32:28.313+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -65> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -64> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -63> 2021-02-04T08:32:28.313+0000 7f26aa8c3700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -62> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -61> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug -60> 2021-02-04T08:32:28.313+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:28 osd07 bash[30988]: debug -59> 2021-02-04T08:32:28.313+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:28 osd07 bash[30988]: debug -58> 2021-02-04T08:32:28.565+0000 7f26ac0c6700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:28 osd07 bash[30988]: debug -57> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:28 osd07 bash[30988]: debug -56> 2021-02-04T08:32:28.605+0000 7f26aa8c3700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:28 osd07 bash[30988]: debug -55> 2021-02-04T08:32:28.605+0000 7f26aa8c3700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:28 osd07 bash[30988]: debug -54> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:28 osd07 bash[30988]: debug -53> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command objecter_requests hook 0x55cd2212a970 Feb 04 09:32:28 osd07 bash[30988]: debug -52> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -51> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -50> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -49> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -48> 2021-02-04T08:32:28.605+0000 7f26b4351f40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:28 osd07 bash[30988]: debug -47> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -46> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -45> 2021-02-04T08:32:28.605+0000 7f26b4351f40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:28 osd07 bash[30988]: debug -44> 2021-02-04T08:32:28.605+0000 7f26b4351f40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:28 osd07 bash[30988]: debug -43> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:28 osd07 bash[30988]: debug -42> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:28 osd07 bash[30988]: debug -41> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -40> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -39> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -38> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -37> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:28 osd07 bash[30988]: debug -36> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -35> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -34> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -33> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -32> 2021-02-04T08:32:28.605+0000 7f26a784c700 10 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -31> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -30> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -29> 2021-02-04T08:32:28.605+0000 7f26a704b700 20 bdev(0x55cd22e6c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -28> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -27> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:28 osd07 bash[30988]: debug -26> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -25> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -24> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -23> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -22> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:28 osd07 bash[30988]: debug -21> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:28 osd07 bash[30988]: debug -20> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:28 osd07 bash[30988]: debug -19> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:28 osd07 bash[30988]: debug -18> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:28 osd07 bash[30988]: debug -17> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluestore bluefs available hook 0x55cd2212a980 Feb 04 09:32:28 osd07 bash[30988]: debug -16> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 asok(0x55cd221d6000) register_command bluefs stats hook 0x55cd2212a980 Feb 04 09:32:28 osd07 bash[30988]: debug -15> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -14> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:28 osd07 bash[30988]: debug -13> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:28 osd07 bash[30988]: debug -12> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:28 osd07 bash[30988]: debug -11> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -10> 2021-02-04T08:32:28.605+0000 7f26b4351f40 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:28 osd07 bash[30988]: debug -9> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:28 osd07 bash[30988]: debug -8> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:28 osd07 bash[30988]: debug -7> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:28 osd07 bash[30988]: debug -6> 2021-02-04T08:32:28.605+0000 7f26a6049700 20 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:28 osd07 bash[30988]: debug -5> 2021-02-04T08:32:28.605+0000 7f26b4351f40 1 bluefs mount Feb 04 09:32:28 osd07 bash[30988]: debug -4> 2021-02-04T08:32:28.605+0000 7f26b4351f40 10 bluefs _open_super Feb 04 09:32:28 osd07 bash[30988]: debug -3> 2021-02-04T08:32:28.605+0000 7f26b4351f40 5 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:28 osd07 bash[30988]: debug -2> 2021-02-04T08:32:28.605+0000 7f26a684a700 10 bdev(0x55cd22e6c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:28 osd07 bash[30988]: debug -1> 2021-02-04T08:32:28.609+0000 7f26b4351f40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f26b4351f40 time 2021-02-04T08:32:28.609047+0000 Feb 04 09:32:28 osd07 bash[30988]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x55cd16fa4372] Feb 04 09:32:28 osd07 bash[30988]: 2: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 4: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 5: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 10: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 11: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 12: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 13: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: debug 0> 2021-02-04T08:32:28.613+0000 7f26b4351f40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:28 osd07 bash[30988]: in thread 7f26b4351f40 thread_name:ceph-osd Feb 04 09:32:28 osd07 bash[30988]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:28 osd07 bash[30988]: 1: (()+0x12b20) [0x7f26b20b9b20] Feb 04 09:32:28 osd07 bash[30988]: 2: (gsignal()+0x10f) [0x7f26b0d217ff] Feb 04 09:32:28 osd07 bash[30988]: 3: (abort()+0x127) [0x7f26b0d0bc35] Feb 04 09:32:28 osd07 bash[30988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x55cd16fa43c3] Feb 04 09:32:28 osd07 bash[30988]: 5: (()+0x50658c) [0x55cd16fa458c] Feb 04 09:32:28 osd07 bash[30988]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x55cd1767cc3d] Feb 04 09:32:28 osd07 bash[30988]: 7: (BlueFS::_open_super()+0x155) [0x55cd1761c0e5] Feb 04 09:32:28 osd07 bash[30988]: 8: (BlueFS::mount()+0xe9) [0x55cd17640f99] Feb 04 09:32:28 osd07 bash[30988]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x55cd1751e974] Feb 04 09:32:28 osd07 bash[30988]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x55cd1751fe1c] Feb 04 09:32:28 osd07 bash[30988]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x55cd17532d6c] Feb 04 09:32:28 osd07 bash[30988]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x55cd1758b007] Feb 04 09:32:28 osd07 bash[30988]: 13: (OSD::init()+0x380) [0x55cd170b2f20] Feb 04 09:32:28 osd07 bash[30988]: 14: (main()+0x47e4) [0x55cd17006264] Feb 04 09:32:28 osd07 bash[30988]: 15: (__libc_start_main()+0xf3) [0x7f26b0d0d7b3] Feb 04 09:32:28 osd07 bash[30988]: 16: (_start()+0x2e) [0x55cd1703456e] Feb 04 09:32:28 osd07 bash[30988]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:28 osd07 bash[30988]: --- logging levels --- Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 none Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 lockdep Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 context Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 crush Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_balancer Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_locker Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_log Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_log_expire Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mds_migrator Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 buffer Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 timer Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 filer Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 striper Feb 04 09:32:28 osd07 bash[30988]: 0/ 1 objecter Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rados Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd_mirror Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd_replay Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 rbd_rwl Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 journaler Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 objectcacher Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 immutable_obj_cache Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 client Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 osd Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 optracker Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 objclass Feb 04 09:32:28 osd07 bash[30988]: 1/ 3 filestore Feb 04 09:32:28 osd07 bash[30988]: 1/ 3 journal Feb 04 09:32:28 osd07 bash[30988]: 0/ 0 ms Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mon Feb 04 09:32:28 osd07 bash[30988]: 0/10 monc Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 paxos Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 tp Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 auth Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 crypto Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 finisher Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 reserver Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 heartbeatmap Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 perfcounter Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 rgw Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 rgw_sync Feb 04 09:32:28 osd07 bash[30988]: 1/10 civetweb Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 javaclient Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 asok Feb 04 09:32:28 osd07 bash[30988]: 1/ 1 throttle Feb 04 09:32:28 osd07 bash[30988]: 0/ 0 refs Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 compressor Feb 04 09:32:28 osd07 bash[30988]: 20/20 bluestore Feb 04 09:32:28 osd07 bash[30988]: 20/20 bluefs Feb 04 09:32:28 osd07 bash[30988]: 20/20 bdev Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 kstore Feb 04 09:32:28 osd07 bash[30988]: 4/ 5 rocksdb Feb 04 09:32:28 osd07 bash[30988]: 4/ 5 leveldb Feb 04 09:32:28 osd07 bash[30988]: 4/ 5 memdb Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 fuse Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mgr Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 mgrc Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 dpdk Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 eventtrace Feb 04 09:32:28 osd07 bash[30988]: 1/ 5 prioritycache Feb 04 09:32:28 osd07 bash[30988]: 0/ 5 test Feb 04 09:32:28 osd07 bash[30988]: -2/-2 (syslog threshold) Feb 04 09:32:28 osd07 bash[30988]: 99/99 (stderr threshold) Feb 04 09:32:28 osd07 bash[30988]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:28 osd07 bash[30988]: 7f26a6049700 / bstore_discard Feb 04 09:32:28 osd07 bash[30988]: 7f26a684a700 / bstore_aio Feb 04 09:32:28 osd07 bash[30988]: 7f26a704b700 / bstore_discard Feb 04 09:32:28 osd07 bash[30988]: 7f26a784c700 / bstore_aio Feb 04 09:32:28 osd07 bash[30988]: 7f26aa8c3700 / OpHistorySvc Feb 04 09:32:28 osd07 bash[30988]: 7f26ab0c4700 / safe_timer Feb 04 09:32:28 osd07 bash[30988]: 7f26ab8c5700 / signal_handler Feb 04 09:32:28 osd07 bash[30988]: 7f26ac0c6700 / ceph-osd Feb 04 09:32:28 osd07 bash[30988]: 7f26ac8c7700 / admin_socket Feb 04 09:32:28 osd07 bash[30988]: 7f26ad0c8700 / service Feb 04 09:32:28 osd07 bash[30988]: 7f26ad8c9700 / msgr-worker-2 Feb 04 09:32:28 osd07 bash[30988]: 7f26ae0ca700 / msgr-worker-1 Feb 04 09:32:28 osd07 bash[30988]: 7f26ae8cb700 / msgr-worker-0 Feb 04 09:32:28 osd07 bash[30988]: 7f26b4351f40 / ceph-osd Feb 04 09:32:28 osd07 bash[30988]: max_recent 10000 Feb 04 09:32:28 osd07 bash[30988]: max_new 1000 Feb 04 09:32:28 osd07 bash[30988]: log_file /var/lib/ceph/crash/2021-02-04T08:32:28.616680Z_fd5f3516-dc60-4c02-adb2-23ae1dd727e1/log Feb 04 09:32:28 osd07 bash[30988]: --- end dump of recent events --- Feb 04 09:32:28 osd07 bash[30988]: reraise_fatal: default handler for signal 6 didn't terminate the process? Feb 04 09:32:28 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Main process exited, code=exited, status=1/FAILURE Feb 04 09:32:28 osd07 bash[31201]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:28 osd07 bash[31201]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-deactivate Feb 04 09:32:28 osd07 bash[31201]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:29 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Failed with result 'exit-code'. Feb 04 09:32:39 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Service RestartSec=10s expired, scheduling restart. Feb 04 09:32:39 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Scheduled restart job, restart counter is at 2. Feb 04 09:32:39 osd07 systemd[1]: Stopped Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:39 osd07 systemd[1]: Starting Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e... Feb 04 09:32:40 osd07 docker[31292]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:40 osd07 docker[31292]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:40 osd07 systemd[1]: Started Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:40 osd07 bash[31308]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:40 osd07 bash[31308]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-activate Feb 04 09:32:40 osd07 bash[31308]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:40 osd07 bash[31308]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:40 osd07 bash[31308]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 --path /var/lib/ceph/osd/ceph-18 --no-mon-config Feb 04 09:32:40 osd07 bash[31308]: Running command: /usr/bin/ln -snf /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:40 osd07 bash[31308]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:40 osd07 bash[31308]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7 Feb 04 09:32:40 osd07 bash[31308]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:40 osd07 bash[31308]: --> ceph-volume lvm activate successful for osd ID: 18 Feb 04 09:32:41 osd07 bash[31308]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:41 osd07 bash[31308]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:41 osd07 bash[31308]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc87c0e1700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc87b8e0700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc87b0df700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev aio_wait 0x5652d6680bb0 done Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.781+0000 7fc87b8e0700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.813+0000 7fc87b0df700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.813+0000 7fc87b0df700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.813+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:41 osd07 bash[31308]: debug 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.029+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc87c0e1700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc87c0e1700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc884b6df40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.069+0000 7fc884b6df40 -1 Falling back to public interface Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 0 load: jerasure load: lrc load: isa Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.081+0000 7fc87b0df700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.085+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.085+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.085+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.085+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.333+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc87b0df700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc87b0df700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc878068700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc877867700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fc884b6df40 time 2021-02-04T08:32:42.368854+0000 Feb 04 09:32:42 osd07 bash[31308]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc876865700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluefs mount Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluefs _open_super Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5652cc01b372] Feb 04 09:32:42 osd07 bash[31308]: 2: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 4: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 5: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 10: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 11: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 12: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 13: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: *** Caught signal (Aborted) ** Feb 04 09:32:42 osd07 bash[31308]: in thread 7fc884b6df40 thread_name:ceph-osd Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.365+0000 7fc877066700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.369+0000 7fc884b6df40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fc884b6df40 time 2021-02-04T08:32:42.368854+0000 Feb 04 09:32:42 osd07 bash[31308]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5652cc01b372] Feb 04 09:32:42 osd07 bash[31308]: 2: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 4: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 5: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 10: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 11: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 12: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 13: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (()+0x12b20) [0x7fc8828d5b20] Feb 04 09:32:42 osd07 bash[31308]: 2: (gsignal()+0x10f) [0x7fc88153d7ff] Feb 04 09:32:42 osd07 bash[31308]: 3: (abort()+0x127) [0x7fc881527c35] Feb 04 09:32:42 osd07 bash[31308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5652cc01b3c3] Feb 04 09:32:42 osd07 bash[31308]: 5: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 13: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 14: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 15: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 16: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: debug 2021-02-04T08:32:42.373+0000 7fc884b6df40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:42 osd07 bash[31308]: in thread 7fc884b6df40 thread_name:ceph-osd Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (()+0x12b20) [0x7fc8828d5b20] Feb 04 09:32:42 osd07 bash[31308]: 2: (gsignal()+0x10f) [0x7fc88153d7ff] Feb 04 09:32:42 osd07 bash[31308]: 3: (abort()+0x127) [0x7fc881527c35] Feb 04 09:32:42 osd07 bash[31308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5652cc01b3c3] Feb 04 09:32:42 osd07 bash[31308]: 5: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 13: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 14: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 15: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 16: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:42 osd07 bash[31308]: --- begin dump of recent events --- Feb 04 09:32:42 osd07 bash[31308]: debug -376> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command assert hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -375> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command abort hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -374> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command leak_some_memory hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -373> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perfcounters_dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -372> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command 1 hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -371> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -370> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perfcounters_schema hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -369> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf histogram dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -368> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command 2 hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -367> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf schema hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -366> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf histogram schema hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -365> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf reset hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -364> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config show hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -363> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config help hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -362> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config set hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -361> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config unset hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -360> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config get hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -359> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config diff hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -358> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config diff get hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -357> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command injectargs hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -356> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command log flush hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -355> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command log dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -354> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command log reopen hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -353> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command dump_mempools hook 0x5652d72cc068 Feb 04 09:32:42 osd07 bash[31308]: debug -352> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: get_monmap_and_config Feb 04 09:32:42 osd07 bash[31308]: debug -351> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: build_initial_monmap Feb 04 09:32:42 osd07 bash[31308]: debug -350> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: monmap: Feb 04 09:32:42 osd07 bash[31308]: epoch 0 Feb 04 09:32:42 osd07 bash[31308]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:42 osd07 bash[31308]: last_changed 2021-02-04T08:32:41.524012+0000 Feb 04 09:32:42 osd07 bash[31308]: created 2021-02-04T08:32:41.524012+0000 Feb 04 09:32:42 osd07 bash[31308]: min_mon_release 0 (unknown) Feb 04 09:32:42 osd07 bash[31308]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:42 osd07 bash[31308]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:42 osd07 bash[31308]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:42 osd07 bash[31308]: debug -349> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -348> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -347> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -346> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -345> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -344> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -343> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -342> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -341> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -340> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -339> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -338> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -337> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -336> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -335> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -334> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -333> 2021-02-04T08:32:41.521+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -332> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: init Feb 04 09:32:42 osd07 bash[31308]: debug -331> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -330> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -329> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -328> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -327> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -326> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -325> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -324> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -323> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -322> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -321> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -320> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -319> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -318> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -317> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -316> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -315> 2021-02-04T08:32:41.521+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -314> 2021-02-04T08:32:41.521+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -313> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: _reopen_session rank -1 Feb 04 09:32:42 osd07 bash[31308]: debug -312> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: _add_conns ranks=[1,0,2] Feb 04 09:32:42 osd07 bash[31308]: debug -311> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): picked mon.noname-b con 0x5652d66f6c00 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:42 osd07 bash[31308]: debug -310> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): picked mon.noname-a con 0x5652d66f7000 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:42 osd07 bash[31308]: debug -309> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): picked mon.noname-c con 0x5652d66f7400 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:42 osd07 bash[31308]: debug -308> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): start opening mon connection Feb 04 09:32:42 osd07 bash[31308]: debug -307> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): start opening mon connection Feb 04 09:32:42 osd07 bash[31308]: debug -306> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): start opening mon connection Feb 04 09:32:42 osd07 bash[31308]: debug -305> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): _renew_subs Feb 04 09:32:42 osd07 bash[31308]: debug -304> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:41.525464+0000 Feb 04 09:32:42 osd07 bash[31308]: debug -303> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): get_auth_request con 0x5652d66f6c00 auth_method 0 Feb 04 09:32:42 osd07 bash[31308]: debug -302> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:42 osd07 bash[31308]: debug -301> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:42 osd07 bash[31308]: debug -300> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): get_auth_request con 0x5652d66f7400 auth_method 0 Feb 04 09:32:42 osd07 bash[31308]: debug -299> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:42 osd07 bash[31308]: debug -298> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:42 osd07 bash[31308]: debug -297> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): get_auth_request con 0x5652d66f7000 auth_method 0 Feb 04 09:32:42 osd07 bash[31308]: debug -296> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:42 osd07 bash[31308]: debug -295> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:42 osd07 bash[31308]: debug -294> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:42 osd07 bash[31308]: debug -293> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:42 osd07 bash[31308]: debug -292> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:42 osd07 bash[31308]: debug -291> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:42 osd07 bash[31308]: debug -290> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:42 osd07 bash[31308]: debug -289> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:42 osd07 bash[31308]: debug -288> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:42 osd07 bash[31308]: debug -287> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:42 osd07 bash[31308]: debug -286> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:42 osd07 bash[31308]: debug -285> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_done global_id 3644930 payload 450 Feb 04 09:32:42 osd07 bash[31308]: debug -284> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient: _finish_hunting 0 Feb 04 09:32:42 osd07 bash[31308]: debug -283> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 1 monclient: found mon.noname-b Feb 04 09:32:42 osd07 bash[31308]: debug -282> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.15:3300/0 Feb 04 09:32:42 osd07 bash[31308]: debug -281> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:42 osd07 bash[31308]: debug -280> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: got monmap 9 from mon.noname-b (according to old e9) Feb 04 09:32:42 osd07 bash[31308]: debug -279> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: dump: Feb 04 09:32:42 osd07 bash[31308]: epoch 9 Feb 04 09:32:42 osd07 bash[31308]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:42 osd07 bash[31308]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:42 osd07 bash[31308]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:42 osd07 bash[31308]: min_mon_release 15 (octopus) Feb 04 09:32:42 osd07 bash[31308]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:42 osd07 bash[31308]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:42 osd07 bash[31308]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:42 osd07 bash[31308]: debug -278> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: _finish_auth 0 Feb 04 09:32:42 osd07 bash[31308]: debug -277> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:32:11.527444+0000) Feb 04 09:32:42 osd07 bash[31308]: debug -276> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: _send_mon_message to mon.osd05 at v2:10.10.0.15:3300/0 Feb 04 09:32:42 osd07 bash[31308]: debug -275> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 monclient: authenticate success, global_id 3644930 Feb 04 09:32:42 osd07 bash[31308]: debug -274> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:42 osd07 bash[31308]: debug -273> 2021-02-04T08:32:41.525+0000 7fc884b6df40 10 monclient: get_monmap_and_config success Feb 04 09:32:42 osd07 bash[31308]: debug -272> 2021-02-04T08:32:41.525+0000 7fc884b6df40 10 monclient: shutdown Feb 04 09:32:42 osd07 bash[31308]: debug -271> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 Feb 04 09:32:42 osd07 bash[31308]: debug -270> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 4 set_mon_vals no callback set Feb 04 09:32:42 osd07 bash[31308]: debug -269> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:42 osd07 bash[31308]: debug -268> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:42 osd07 bash[31308]: debug -267> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:42 osd07 bash[31308]: debug -266> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:42 osd07 bash[31308]: debug -265> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:42 osd07 bash[31308]: debug -264> 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:42 osd07 bash[31308]: debug -263> 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:42 osd07 bash[31308]: debug -262> 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:42 osd07 bash[31308]: debug -261> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:42 osd07 bash[31308]: debug -260> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:42 osd07 bash[31308]: debug -259> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command 0 hook 0x5652d6650388 Feb 04 09:32:42 osd07 bash[31308]: debug -258> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command version hook 0x5652d6650388 Feb 04 09:32:42 osd07 bash[31308]: debug -257> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command git_version hook 0x5652d6650388 Feb 04 09:32:42 osd07 bash[31308]: debug -256> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command help hook 0x5652d664c500 Feb 04 09:32:42 osd07 bash[31308]: debug -255> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command get_command_descriptions hook 0x5652d664c070 Feb 04 09:32:42 osd07 bash[31308]: debug -254> 2021-02-04T08:32:41.525+0000 7fc87d0e3700 5 asok(0x5652d66f8000) entry start Feb 04 09:32:42 osd07 bash[31308]: debug -253> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 object store type is bluestore Feb 04 09:32:42 osd07 bash[31308]: debug -252> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:42 osd07 bash[31308]: debug -251> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -250> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -249> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -248> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -247> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -246> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -245> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -244> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -243> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -242> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -241> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -240> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -239> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:42 osd07 bash[31308]: debug -238> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -237> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -236> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -235> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -234> 2021-02-04T08:32:41.529+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -233> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -232> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -231> 2021-02-04T08:32:41.529+0000 7fc87c0e1700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -230> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -229> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug -228> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluestore bluefs available hook 0x5652d664c0b0 Feb 04 09:32:42 osd07 bash[31308]: debug -227> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluefs stats hook 0x5652d664c0b0 Feb 04 09:32:42 osd07 bash[31308]: debug -226> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -225> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -224> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -223> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -222> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -221> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -220> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -219> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -218> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:42 osd07 bash[31308]: debug -217> 2021-02-04T08:32:41.529+0000 7fc87b8e0700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -216> 2021-02-04T08:32:41.529+0000 7fc87b0df700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -215> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) unregister_commands bluefs stats Feb 04 09:32:42 osd07 bash[31308]: debug -214> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) unregister_commands bluestore bluefs available Feb 04 09:32:42 osd07 bash[31308]: debug -213> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev aio_wait 0x5652d6680bb0 done Feb 04 09:32:42 osd07 bash[31308]: debug -212> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug -211> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug -210> 2021-02-04T08:32:41.781+0000 7fc87b8e0700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug -209> 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug -208> 2021-02-04T08:32:41.813+0000 7fc87b0df700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug -207> 2021-02-04T08:32:41.813+0000 7fc87b0df700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug -206> 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug -205> 2021-02-04T08:32:41.813+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug -204> 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug -203> 2021-02-04T08:32:42.029+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug -202> 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug -201> 2021-02-04T08:32:42.069+0000 7fc87c0e1700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug -200> 2021-02-04T08:32:42.069+0000 7fc87c0e1700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug -199> 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug -198> 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:42 osd07 bash[31308]: debug -197> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -196> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -195> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -194> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -193> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -192> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -191> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -190> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -189> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -188> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -187> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -186> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -185> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -184> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -183> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -182> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -181> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -180> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -179> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -178> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -177> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -176> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -175> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -174> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -173> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -172> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -171> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -170> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -169> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -168> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -167> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -166> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -165> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -164> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -163> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -162> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -161> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -160> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -159> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -158> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -157> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -156> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -155> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -154> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -153> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -152> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -151> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -150> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -149> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -148> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -147> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -146> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -145> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -144> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -143> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -142> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -141> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -140> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -139> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -138> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -137> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -136> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -135> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -134> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -133> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -132> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -131> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -130> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -129> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -128> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -127> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -126> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -125> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -124> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -123> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -122> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -121> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -120> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -119> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -118> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -117> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -116> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -115> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -114> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -113> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -112> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -111> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -110> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -109> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -108> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -107> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -106> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -105> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -104> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -103> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -102> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -101> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -100> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -99> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -98> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -97> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -96> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -95> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -94> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -93> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -92> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -91> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -90> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -89> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -88> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -87> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -86> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -85> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -84> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -83> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -82> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -81> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -80> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -79> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -78> 2021-02-04T08:32:42.069+0000 7fc884b6df40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:42 osd07 bash[31308]: debug -77> 2021-02-04T08:32:42.069+0000 7fc884b6df40 -1 Falling back to public interface Feb 04 09:32:42 osd07 bash[31308]: debug -76> 2021-02-04T08:32:42.073+0000 7fc884b6df40 10 monclient: build_initial_monmap Feb 04 09:32:42 osd07 bash[31308]: debug -75> 2021-02-04T08:32:42.073+0000 7fc884b6df40 10 monclient: monmap: Feb 04 09:32:42 osd07 bash[31308]: epoch 0 Feb 04 09:32:42 osd07 bash[31308]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:42 osd07 bash[31308]: last_changed 0.000000 Feb 04 09:32:42 osd07 bash[31308]: created 0.000000 Feb 04 09:32:42 osd07 bash[31308]: min_mon_release 0 (unknown) Feb 04 09:32:42 osd07 bash[31308]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:42 osd07 bash[31308]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:42 osd07 bash[31308]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:42 osd07 bash[31308]: debug -74> 2021-02-04T08:32:42.081+0000 7fc884b6df40 0 load: jerasure load: lrc load: isa Feb 04 09:32:42 osd07 bash[31308]: debug -73> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -72> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -71> 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:42 osd07 bash[31308]: debug -70> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -69> 2021-02-04T08:32:42.081+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -68> 2021-02-04T08:32:42.081+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -67> 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -66> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -65> 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -64> 2021-02-04T08:32:42.081+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -63> 2021-02-04T08:32:42.081+0000 7fc87b0df700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -62> 2021-02-04T08:32:42.085+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -61> 2021-02-04T08:32:42.085+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug -60> 2021-02-04T08:32:42.085+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug -59> 2021-02-04T08:32:42.085+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug -58> 2021-02-04T08:32:42.333+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug -57> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug -56> 2021-02-04T08:32:42.365+0000 7fc87b0df700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug -55> 2021-02-04T08:32:42.365+0000 7fc87b0df700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug -54> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug -53> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command objecter_requests hook 0x5652d664c980 Feb 04 09:32:42 osd07 bash[31308]: debug -52> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -51> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -50> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -49> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -48> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -47> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -46> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -45> 2021-02-04T08:32:42.365+0000 7fc884b6df40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:42 osd07 bash[31308]: debug -44> 2021-02-04T08:32:42.365+0000 7fc884b6df40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:42 osd07 bash[31308]: debug -43> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:42 osd07 bash[31308]: debug -42> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:42 osd07 bash[31308]: debug -41> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -40> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -39> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -38> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -37> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:42 osd07 bash[31308]: debug -36> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -35> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -34> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -33> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -32> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -31> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -30> 2021-02-04T08:32:42.365+0000 7fc878068700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -29> 2021-02-04T08:32:42.365+0000 7fc877867700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -28> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -27> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug -26> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -25> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -24> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -23> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -22> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:42 osd07 bash[31308]: debug -21> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -20> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -19> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:42 osd07 bash[31308]: debug -18> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:42 osd07 bash[31308]: debug -17> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluestore bluefs available hook 0x5652d664c990 Feb 04 09:32:42 osd07 bash[31308]: debug -16> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluefs stats hook 0x5652d664c990 Feb 04 09:32:42 osd07 bash[31308]: debug -15> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -14> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -13> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -12> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -11> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -10> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -9> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -8> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -7> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:42 osd07 bash[31308]: debug -6> 2021-02-04T08:32:42.365+0000 7fc876865700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -5> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluefs mount Feb 04 09:32:42 osd07 bash[31308]: debug -4> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluefs _open_super Feb 04 09:32:42 osd07 bash[31308]: debug -3> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:42 osd07 bash[31308]: debug -2> 2021-02-04T08:32:42.365+0000 7fc877066700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -1> 2021-02-04T08:32:42.369+0000 7fc884b6df40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fc884b6df40 time 2021-02-04T08:32:42.368854+0000 Feb 04 09:32:42 osd07 bash[31308]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5652cc01b372] Feb 04 09:32:42 osd07 bash[31308]: 2: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 4: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 5: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 10: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 11: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 12: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 13: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: debug 0> 2021-02-04T08:32:42.373+0000 7fc884b6df40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:42 osd07 bash[31308]: in thread 7fc884b6df40 thread_name:ceph-osd Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (()+0x12b20) [0x7fc8828d5b20] Feb 04 09:32:42 osd07 bash[31308]: 2: (gsignal()+0x10f) [0x7fc88153d7ff] Feb 04 09:32:42 osd07 bash[31308]: 3: (abort()+0x127) [0x7fc881527c35] Feb 04 09:32:42 osd07 bash[31308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5652cc01b3c3] Feb 04 09:32:42 osd07 bash[31308]: 5: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 13: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 14: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 15: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 16: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:42 osd07 bash[31308]: --- logging levels --- Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 none Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 lockdep Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 context Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 crush Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_balancer Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_locker Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_log Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_log_expire Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_migrator Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 buffer Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 timer Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 filer Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 striper Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 objecter Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rados Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd_mirror Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd_replay Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd_rwl Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 journaler Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 objectcacher Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 immutable_obj_cache Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 client Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 osd Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 optracker Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 objclass Feb 04 09:32:42 osd07 bash[31308]: 1/ 3 filestore Feb 04 09:32:42 osd07 bash[31308]: 1/ 3 journal Feb 04 09:32:42 osd07 bash[31308]: 0/ 0 ms Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mon Feb 04 09:32:42 osd07 bash[31308]: 0/10 monc Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 paxos Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 tp Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 auth Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 crypto Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 finisher Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 reserver Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 heartbeatmap Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 perfcounter Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 rgw Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 rgw_sync Feb 04 09:32:42 osd07 bash[31308]: 1/10 civetweb Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 javaclient Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 asok Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 throttle Feb 04 09:32:42 osd07 bash[31308]: 0/ 0 refs Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 compressor Feb 04 09:32:42 osd07 bash[31308]: 20/20 bluestore Feb 04 09:32:42 osd07 bash[31308]: 20/20 bluefs Feb 04 09:32:42 osd07 bash[31308]: 20/20 bdev Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 kstore Feb 04 09:32:42 osd07 bash[31308]: 4/ 5 rocksdb Feb 04 09:32:42 osd07 bash[31308]: 4/ 5 leveldb Feb 04 09:32:42 osd07 bash[31308]: 4/ 5 memdb Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 fuse Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mgr Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mgrc Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 dpdk Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 eventtrace Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 prioritycache Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 test Feb 04 09:32:42 osd07 bash[31308]: -2/-2 (syslog threshold) Feb 04 09:32:42 osd07 bash[31308]: 99/99 (stderr threshold) Feb 04 09:32:42 osd07 bash[31308]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:42 osd07 bash[31308]: 7fc876865700 / bstore_discard Feb 04 09:32:42 osd07 bash[31308]: 7fc877066700 / bstore_aio Feb 04 09:32:42 osd07 bash[31308]: 7fc877867700 / bstore_discard Feb 04 09:32:42 osd07 bash[31308]: 7fc878068700 / bstore_aio Feb 04 09:32:42 osd07 bash[31308]: 7fc87b0df700 / OpHistorySvc Feb 04 09:32:42 osd07 bash[31308]: 7fc87b8e0700 / safe_timer Feb 04 09:32:42 osd07 bash[31308]: 7fc87c0e1700 / signal_handler Feb 04 09:32:42 osd07 bash[31308]: 7fc87c8e2700 / ceph-osd Feb 04 09:32:42 osd07 bash[31308]: 7fc87d0e3700 / admin_socket Feb 04 09:32:42 osd07 bash[31308]: 7fc87d8e4700 / service Feb 04 09:32:42 osd07 bash[31308]: 7fc87e0e5700 / msgr-worker-2 Feb 04 09:32:42 osd07 bash[31308]: 7fc87e8e6700 / msgr-worker-1 Feb 04 09:32:42 osd07 bash[31308]: 7fc87f0e7700 / msgr-worker-0 Feb 04 09:32:42 osd07 bash[31308]: 7fc884b6df40 / ceph-osd Feb 04 09:32:42 osd07 bash[31308]: max_recent 10000 Feb 04 09:32:42 osd07 bash[31308]: max_new 1000 Feb 04 09:32:42 osd07 bash[31308]: log_file Feb 04 09:32:42 osd07 bash[31308]: --- end dump of recent events --- Feb 04 09:32:42 osd07 bash[31308]: --- begin dump of recent events --- Feb 04 09:32:42 osd07 bash[31308]: debug -376> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command assert hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -375> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command abort hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -374> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command leak_some_memory hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -373> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perfcounters_dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -372> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command 1 hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -371> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -370> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perfcounters_schema hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -369> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf histogram dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -368> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command 2 hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -367> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf schema hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -366> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf histogram schema hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -365> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command perf reset hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -364> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config show hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -363> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config help hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -362> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config set hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -361> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config unset hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -360> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config get hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -359> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config diff hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -358> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command config diff get hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -357> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command injectargs hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -356> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command log flush hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -355> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command log dump hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -354> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command log reopen hook 0x5652d664c5b0 Feb 04 09:32:42 osd07 bash[31308]: debug -353> 2021-02-04T08:32:41.517+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command dump_mempools hook 0x5652d72cc068 Feb 04 09:32:42 osd07 bash[31308]: debug -352> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: get_monmap_and_config Feb 04 09:32:42 osd07 bash[31308]: debug -351> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: build_initial_monmap Feb 04 09:32:42 osd07 bash[31308]: debug -350> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: monmap: Feb 04 09:32:42 osd07 bash[31308]: epoch 0 Feb 04 09:32:42 osd07 bash[31308]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:42 osd07 bash[31308]: last_changed 2021-02-04T08:32:41.524012+0000 Feb 04 09:32:42 osd07 bash[31308]: created 2021-02-04T08:32:41.524012+0000 Feb 04 09:32:42 osd07 bash[31308]: min_mon_release 0 (unknown) Feb 04 09:32:42 osd07 bash[31308]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:42 osd07 bash[31308]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:42 osd07 bash[31308]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:42 osd07 bash[31308]: debug -349> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -348> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -347> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -346> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -345> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -344> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -343> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -342> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -341> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -340> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -339> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -338> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -337> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -336> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -335> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -334> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e2940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -333> 2021-02-04T08:32:41.521+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -332> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: init Feb 04 09:32:42 osd07 bash[31308]: debug -331> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -330> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -329> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -328> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -327> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -326> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -325> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -324> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -323> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -322> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -321> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -320> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -319> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -318> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -317> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -316> 2021-02-04T08:32:41.521+0000 7fc884b6df40 5 AuthRegistry(0x7ffc5adb6430) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -315> 2021-02-04T08:32:41.521+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -314> 2021-02-04T08:32:41.521+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -313> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: _reopen_session rank -1 Feb 04 09:32:42 osd07 bash[31308]: debug -312> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient: _add_conns ranks=[1,0,2] Feb 04 09:32:42 osd07 bash[31308]: debug -311> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): picked mon.noname-b con 0x5652d66f6c00 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:42 osd07 bash[31308]: debug -310> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): picked mon.noname-a con 0x5652d66f7000 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:42 osd07 bash[31308]: debug -309> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): picked mon.noname-c con 0x5652d66f7400 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:42 osd07 bash[31308]: debug -308> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): start opening mon connection Feb 04 09:32:42 osd07 bash[31308]: debug -307> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): start opening mon connection Feb 04 09:32:42 osd07 bash[31308]: debug -306> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): start opening mon connection Feb 04 09:32:42 osd07 bash[31308]: debug -305> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): _renew_subs Feb 04 09:32:42 osd07 bash[31308]: debug -304> 2021-02-04T08:32:41.521+0000 7fc884b6df40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:41.525464+0000 Feb 04 09:32:42 osd07 bash[31308]: debug -303> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): get_auth_request con 0x5652d66f6c00 auth_method 0 Feb 04 09:32:42 osd07 bash[31308]: debug -302> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:42 osd07 bash[31308]: debug -301> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:42 osd07 bash[31308]: debug -300> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): get_auth_request con 0x5652d66f7400 auth_method 0 Feb 04 09:32:42 osd07 bash[31308]: debug -299> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:42 osd07 bash[31308]: debug -298> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:42 osd07 bash[31308]: debug -297> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): get_auth_request con 0x5652d66f7000 auth_method 0 Feb 04 09:32:42 osd07 bash[31308]: debug -296> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:42 osd07 bash[31308]: debug -295> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:42 osd07 bash[31308]: debug -294> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:42 osd07 bash[31308]: debug -293> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:42 osd07 bash[31308]: debug -292> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:42 osd07 bash[31308]: debug -291> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:42 osd07 bash[31308]: debug -290> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:42 osd07 bash[31308]: debug -289> 2021-02-04T08:32:41.525+0000 7fc87f0e7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:42 osd07 bash[31308]: debug -288> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:42 osd07 bash[31308]: debug -287> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:42 osd07 bash[31308]: debug -286> 2021-02-04T08:32:41.525+0000 7fc87e0e5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:42 osd07 bash[31308]: debug -285> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient(hunting): handle_auth_done global_id 3644930 payload 450 Feb 04 09:32:42 osd07 bash[31308]: debug -284> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient: _finish_hunting 0 Feb 04 09:32:42 osd07 bash[31308]: debug -283> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 1 monclient: found mon.noname-b Feb 04 09:32:42 osd07 bash[31308]: debug -282> 2021-02-04T08:32:41.525+0000 7fc87e8e6700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.15:3300/0 Feb 04 09:32:42 osd07 bash[31308]: debug -281> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:42 osd07 bash[31308]: debug -280> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: got monmap 9 from mon.noname-b (according to old e9) Feb 04 09:32:42 osd07 bash[31308]: debug -279> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: dump: Feb 04 09:32:42 osd07 bash[31308]: epoch 9 Feb 04 09:32:42 osd07 bash[31308]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:42 osd07 bash[31308]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:42 osd07 bash[31308]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:42 osd07 bash[31308]: min_mon_release 15 (octopus) Feb 04 09:32:42 osd07 bash[31308]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:42 osd07 bash[31308]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:42 osd07 bash[31308]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:42 osd07 bash[31308]: debug -278> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: _finish_auth 0 Feb 04 09:32:42 osd07 bash[31308]: debug -277> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:32:11.527444+0000) Feb 04 09:32:42 osd07 bash[31308]: debug -276> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: _send_mon_message to mon.osd05 at v2:10.10.0.15:3300/0 Feb 04 09:32:42 osd07 bash[31308]: debug -275> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 monclient: authenticate success, global_id 3644930 Feb 04 09:32:42 osd07 bash[31308]: debug -274> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:42 osd07 bash[31308]: debug -273> 2021-02-04T08:32:41.525+0000 7fc884b6df40 10 monclient: get_monmap_and_config success Feb 04 09:32:42 osd07 bash[31308]: debug -272> 2021-02-04T08:32:41.525+0000 7fc884b6df40 10 monclient: shutdown Feb 04 09:32:42 osd07 bash[31308]: debug -271> 2021-02-04T08:32:41.525+0000 7fc87d8e4700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 Feb 04 09:32:42 osd07 bash[31308]: debug -270> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 4 set_mon_vals no callback set Feb 04 09:32:42 osd07 bash[31308]: debug -269> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:42 osd07 bash[31308]: debug -268> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:42 osd07 bash[31308]: debug -267> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:42 osd07 bash[31308]: debug -266> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:42 osd07 bash[31308]: debug -265> 2021-02-04T08:32:41.525+0000 7fc87c0e1700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:42 osd07 bash[31308]: debug -264> 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:42 osd07 bash[31308]: debug -263> 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:42 osd07 bash[31308]: debug -262> 2021-02-04T08:32:41.525+0000 7fc884b6df40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:42 osd07 bash[31308]: debug -261> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:42 osd07 bash[31308]: debug -260> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:42 osd07 bash[31308]: debug -259> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command 0 hook 0x5652d6650388 Feb 04 09:32:42 osd07 bash[31308]: debug -258> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command version hook 0x5652d6650388 Feb 04 09:32:42 osd07 bash[31308]: debug -257> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command git_version hook 0x5652d6650388 Feb 04 09:32:42 osd07 bash[31308]: debug -256> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command help hook 0x5652d664c500 Feb 04 09:32:42 osd07 bash[31308]: debug -255> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command get_command_descriptions hook 0x5652d664c070 Feb 04 09:32:42 osd07 bash[31308]: debug -254> 2021-02-04T08:32:41.525+0000 7fc87d0e3700 5 asok(0x5652d66f8000) entry start Feb 04 09:32:42 osd07 bash[31308]: debug -253> 2021-02-04T08:32:41.525+0000 7fc884b6df40 5 object store type is bluestore Feb 04 09:32:42 osd07 bash[31308]: debug -252> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:42 osd07 bash[31308]: debug -251> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -250> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -249> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -248> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -247> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -246> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -245> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -244> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -243> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -242> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -241> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -240> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -239> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:42 osd07 bash[31308]: debug -238> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -237> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -236> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -235> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -234> 2021-02-04T08:32:41.529+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -233> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -232> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -231> 2021-02-04T08:32:41.529+0000 7fc87c0e1700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -230> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -229> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug -228> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluestore bluefs available hook 0x5652d664c0b0 Feb 04 09:32:42 osd07 bash[31308]: debug -227> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluefs stats hook 0x5652d664c0b0 Feb 04 09:32:42 osd07 bash[31308]: debug -226> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -225> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -224> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -223> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -222> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -221> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -220> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -219> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -218> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:42 osd07 bash[31308]: debug -217> 2021-02-04T08:32:41.529+0000 7fc87b8e0700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -216> 2021-02-04T08:32:41.529+0000 7fc87b0df700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -215> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) unregister_commands bluefs stats Feb 04 09:32:42 osd07 bash[31308]: debug -214> 2021-02-04T08:32:41.529+0000 7fc884b6df40 5 asok(0x5652d66f8000) unregister_commands bluestore bluefs available Feb 04 09:32:42 osd07 bash[31308]: debug -213> 2021-02-04T08:32:41.529+0000 7fc884b6df40 20 bdev aio_wait 0x5652d6680bb0 done Feb 04 09:32:42 osd07 bash[31308]: debug -212> 2021-02-04T08:32:41.529+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug -211> 2021-02-04T08:32:41.529+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug -210> 2021-02-04T08:32:41.781+0000 7fc87b8e0700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug -209> 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug -208> 2021-02-04T08:32:41.813+0000 7fc87b0df700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug -207> 2021-02-04T08:32:41.813+0000 7fc87b0df700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug -206> 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug -205> 2021-02-04T08:32:41.813+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug -204> 2021-02-04T08:32:41.813+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug -203> 2021-02-04T08:32:42.029+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug -202> 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug -201> 2021-02-04T08:32:42.069+0000 7fc87c0e1700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug -200> 2021-02-04T08:32:42.069+0000 7fc87c0e1700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug -199> 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug -198> 2021-02-04T08:32:42.069+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:42 osd07 bash[31308]: debug -197> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -196> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -195> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -194> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -193> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -192> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -191> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -190> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -189> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -188> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -187> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -186> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -185> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -184> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -183> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -182> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e3940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -181> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -180> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -179> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -178> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -177> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -176> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -175> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -174> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -173> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -172> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -171> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -170> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -169> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -168> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -167> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -166> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -165> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e4940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -164> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -163> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -162> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -161> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -160> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -159> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -158> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -157> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -156> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -155> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -154> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -153> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -152> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -151> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -150> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -149> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -148> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d66e5940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -147> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -146> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -145> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -144> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -143> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -142> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -141> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -140> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -139> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -138> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -137> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -136> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -135> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -134> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -133> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -132> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -131> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739c940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -130> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -129> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -128> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -127> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -126> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -125> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -124> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -123> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -122> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -121> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -120> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -119> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -118> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -117> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -116> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -115> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -114> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739d940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -113> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -112> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -111> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -110> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -109> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -108> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -107> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -106> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -105> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -104> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -103> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -102> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -101> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -100> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -99> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -98> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -97> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739e940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -96> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -95> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -94> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -93> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: cephx Feb 04 09:32:42 osd07 bash[31308]: debug -92> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding auth protocol: none Feb 04 09:32:42 osd07 bash[31308]: debug -91> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -90> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -89> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -88> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -87> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -86> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -85> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -84> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -83> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -82> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -81> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: crc Feb 04 09:32:42 osd07 bash[31308]: debug -80> 2021-02-04T08:32:42.069+0000 7fc884b6df40 5 AuthRegistry(0x5652d739f940) adding con mode: secure Feb 04 09:32:42 osd07 bash[31308]: debug -79> 2021-02-04T08:32:42.069+0000 7fc884b6df40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:42 osd07 bash[31308]: debug -78> 2021-02-04T08:32:42.069+0000 7fc884b6df40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:42 osd07 bash[31308]: debug -77> 2021-02-04T08:32:42.069+0000 7fc884b6df40 -1 Falling back to public interface Feb 04 09:32:42 osd07 bash[31308]: debug -76> 2021-02-04T08:32:42.073+0000 7fc884b6df40 10 monclient: build_initial_monmap Feb 04 09:32:42 osd07 bash[31308]: debug -75> 2021-02-04T08:32:42.073+0000 7fc884b6df40 10 monclient: monmap: Feb 04 09:32:42 osd07 bash[31308]: epoch 0 Feb 04 09:32:42 osd07 bash[31308]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:42 osd07 bash[31308]: last_changed 0.000000 Feb 04 09:32:42 osd07 bash[31308]: created 0.000000 Feb 04 09:32:42 osd07 bash[31308]: min_mon_release 0 (unknown) Feb 04 09:32:42 osd07 bash[31308]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:42 osd07 bash[31308]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:42 osd07 bash[31308]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:42 osd07 bash[31308]: debug -74> 2021-02-04T08:32:42.081+0000 7fc884b6df40 0 load: jerasure load: lrc load: isa Feb 04 09:32:42 osd07 bash[31308]: debug -73> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -72> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -71> 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:42 osd07 bash[31308]: debug -70> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -69> 2021-02-04T08:32:42.081+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -68> 2021-02-04T08:32:42.081+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -67> 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -66> 2021-02-04T08:32:42.081+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -65> 2021-02-04T08:32:42.081+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -64> 2021-02-04T08:32:42.081+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -63> 2021-02-04T08:32:42.081+0000 7fc87b0df700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -62> 2021-02-04T08:32:42.085+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -61> 2021-02-04T08:32:42.085+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug -60> 2021-02-04T08:32:42.085+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:42 osd07 bash[31308]: debug -59> 2021-02-04T08:32:42.085+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:42 osd07 bash[31308]: debug -58> 2021-02-04T08:32:42.333+0000 7fc87c8e2700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:42 osd07 bash[31308]: debug -57> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:42 osd07 bash[31308]: debug -56> 2021-02-04T08:32:42.365+0000 7fc87b0df700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:42 osd07 bash[31308]: debug -55> 2021-02-04T08:32:42.365+0000 7fc87b0df700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:42 osd07 bash[31308]: debug -54> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:42 osd07 bash[31308]: debug -53> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command objecter_requests hook 0x5652d664c980 Feb 04 09:32:42 osd07 bash[31308]: debug -52> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -51> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -50> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -49> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -48> 2021-02-04T08:32:42.365+0000 7fc884b6df40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:42 osd07 bash[31308]: debug -47> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -46> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -45> 2021-02-04T08:32:42.365+0000 7fc884b6df40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:42 osd07 bash[31308]: debug -44> 2021-02-04T08:32:42.365+0000 7fc884b6df40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:42 osd07 bash[31308]: debug -43> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:42 osd07 bash[31308]: debug -42> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:42 osd07 bash[31308]: debug -41> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -40> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -39> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -38> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -37> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:42 osd07 bash[31308]: debug -36> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -35> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -34> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -33> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -32> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -31> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -30> 2021-02-04T08:32:42.365+0000 7fc878068700 10 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -29> 2021-02-04T08:32:42.365+0000 7fc877867700 20 bdev(0x5652d7378000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -28> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -27> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:42 osd07 bash[31308]: debug -26> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -25> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -24> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -23> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -22> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:42 osd07 bash[31308]: debug -21> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:42 osd07 bash[31308]: debug -20> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:42 osd07 bash[31308]: debug -19> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:42 osd07 bash[31308]: debug -18> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:42 osd07 bash[31308]: debug -17> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluestore bluefs available hook 0x5652d664c990 Feb 04 09:32:42 osd07 bash[31308]: debug -16> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 asok(0x5652d66f8000) register_command bluefs stats hook 0x5652d664c990 Feb 04 09:32:42 osd07 bash[31308]: debug -15> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -14> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:42 osd07 bash[31308]: debug -13> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:42 osd07 bash[31308]: debug -12> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:42 osd07 bash[31308]: debug -11> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -10> 2021-02-04T08:32:42.365+0000 7fc884b6df40 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:42 osd07 bash[31308]: debug -9> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:42 osd07 bash[31308]: debug -8> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:42 osd07 bash[31308]: debug -7> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:42 osd07 bash[31308]: debug -6> 2021-02-04T08:32:42.365+0000 7fc876865700 20 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:42 osd07 bash[31308]: debug -5> 2021-02-04T08:32:42.365+0000 7fc884b6df40 1 bluefs mount Feb 04 09:32:42 osd07 bash[31308]: debug -4> 2021-02-04T08:32:42.365+0000 7fc884b6df40 10 bluefs _open_super Feb 04 09:32:42 osd07 bash[31308]: debug -3> 2021-02-04T08:32:42.365+0000 7fc884b6df40 5 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:42 osd07 bash[31308]: debug -2> 2021-02-04T08:32:42.365+0000 7fc877066700 10 bdev(0x5652d7378700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:42 osd07 bash[31308]: debug -1> 2021-02-04T08:32:42.369+0000 7fc884b6df40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fc884b6df40 time 2021-02-04T08:32:42.368854+0000 Feb 04 09:32:42 osd07 bash[31308]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5652cc01b372] Feb 04 09:32:42 osd07 bash[31308]: 2: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 4: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 5: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 10: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 11: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 12: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 13: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: debug 0> 2021-02-04T08:32:42.373+0000 7fc884b6df40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:42 osd07 bash[31308]: in thread 7fc884b6df40 thread_name:ceph-osd Feb 04 09:32:42 osd07 bash[31308]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:42 osd07 bash[31308]: 1: (()+0x12b20) [0x7fc8828d5b20] Feb 04 09:32:42 osd07 bash[31308]: 2: (gsignal()+0x10f) [0x7fc88153d7ff] Feb 04 09:32:42 osd07 bash[31308]: 3: (abort()+0x127) [0x7fc881527c35] Feb 04 09:32:42 osd07 bash[31308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5652cc01b3c3] Feb 04 09:32:42 osd07 bash[31308]: 5: (()+0x50658c) [0x5652cc01b58c] Feb 04 09:32:42 osd07 bash[31308]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5652cc6f3c3d] Feb 04 09:32:42 osd07 bash[31308]: 7: (BlueFS::_open_super()+0x155) [0x5652cc6930e5] Feb 04 09:32:42 osd07 bash[31308]: 8: (BlueFS::mount()+0xe9) [0x5652cc6b7f99] Feb 04 09:32:42 osd07 bash[31308]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5652cc595974] Feb 04 09:32:42 osd07 bash[31308]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5652cc596e1c] Feb 04 09:32:42 osd07 bash[31308]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5652cc5a9d6c] Feb 04 09:32:42 osd07 bash[31308]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5652cc602007] Feb 04 09:32:42 osd07 bash[31308]: 13: (OSD::init()+0x380) [0x5652cc129f20] Feb 04 09:32:42 osd07 bash[31308]: 14: (main()+0x47e4) [0x5652cc07d264] Feb 04 09:32:42 osd07 bash[31308]: 15: (__libc_start_main()+0xf3) [0x7fc8815297b3] Feb 04 09:32:42 osd07 bash[31308]: 16: (_start()+0x2e) [0x5652cc0ab56e] Feb 04 09:32:42 osd07 bash[31308]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:42 osd07 bash[31308]: --- logging levels --- Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 none Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 lockdep Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 context Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 crush Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_balancer Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_locker Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_log Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_log_expire Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mds_migrator Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 buffer Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 timer Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 filer Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 striper Feb 04 09:32:42 osd07 bash[31308]: 0/ 1 objecter Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rados Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd_mirror Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd_replay Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 rbd_rwl Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 journaler Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 objectcacher Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 immutable_obj_cache Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 client Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 osd Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 optracker Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 objclass Feb 04 09:32:42 osd07 bash[31308]: 1/ 3 filestore Feb 04 09:32:42 osd07 bash[31308]: 1/ 3 journal Feb 04 09:32:42 osd07 bash[31308]: 0/ 0 ms Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mon Feb 04 09:32:42 osd07 bash[31308]: 0/10 monc Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 paxos Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 tp Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 auth Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 crypto Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 finisher Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 reserver Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 heartbeatmap Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 perfcounter Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 rgw Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 rgw_sync Feb 04 09:32:42 osd07 bash[31308]: 1/10 civetweb Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 javaclient Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 asok Feb 04 09:32:42 osd07 bash[31308]: 1/ 1 throttle Feb 04 09:32:42 osd07 bash[31308]: 0/ 0 refs Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 compressor Feb 04 09:32:42 osd07 bash[31308]: 20/20 bluestore Feb 04 09:32:42 osd07 bash[31308]: 20/20 bluefs Feb 04 09:32:42 osd07 bash[31308]: 20/20 bdev Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 kstore Feb 04 09:32:42 osd07 bash[31308]: 4/ 5 rocksdb Feb 04 09:32:42 osd07 bash[31308]: 4/ 5 leveldb Feb 04 09:32:42 osd07 bash[31308]: 4/ 5 memdb Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 fuse Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mgr Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 mgrc Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 dpdk Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 eventtrace Feb 04 09:32:42 osd07 bash[31308]: 1/ 5 prioritycache Feb 04 09:32:42 osd07 bash[31308]: 0/ 5 test Feb 04 09:32:42 osd07 bash[31308]: -2/-2 (syslog threshold) Feb 04 09:32:42 osd07 bash[31308]: 99/99 (stderr threshold) Feb 04 09:32:42 osd07 bash[31308]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:42 osd07 bash[31308]: 7fc876865700 / bstore_discard Feb 04 09:32:42 osd07 bash[31308]: 7fc877066700 / bstore_aio Feb 04 09:32:42 osd07 bash[31308]: 7fc877867700 / bstore_discard Feb 04 09:32:42 osd07 bash[31308]: 7fc878068700 / bstore_aio Feb 04 09:32:42 osd07 bash[31308]: 7fc87b0df700 / OpHistorySvc Feb 04 09:32:42 osd07 bash[31308]: 7fc87b8e0700 / safe_timer Feb 04 09:32:42 osd07 bash[31308]: 7fc87c0e1700 / signal_handler Feb 04 09:32:42 osd07 bash[31308]: 7fc87c8e2700 / ceph-osd Feb 04 09:32:42 osd07 bash[31308]: 7fc87d0e3700 / admin_socket Feb 04 09:32:42 osd07 bash[31308]: 7fc87d8e4700 / service Feb 04 09:32:42 osd07 bash[31308]: 7fc87e0e5700 / msgr-worker-2 Feb 04 09:32:42 osd07 bash[31308]: 7fc87e8e6700 / msgr-worker-1 Feb 04 09:32:42 osd07 bash[31308]: 7fc87f0e7700 / msgr-worker-0 Feb 04 09:32:42 osd07 bash[31308]: 7fc884b6df40 / ceph-osd Feb 04 09:32:42 osd07 bash[31308]: max_recent 10000 Feb 04 09:32:42 osd07 bash[31308]: max_new 1000 Feb 04 09:32:42 osd07 bash[31308]: log_file /var/lib/ceph/crash/2021-02-04T08:32:42.376593Z_11d6a591-712a-421a-ba19-be1088c1f20d/log Feb 04 09:32:42 osd07 bash[31308]: --- end dump of recent events --- Feb 04 09:32:42 osd07 bash[31308]: reraise_fatal: default handler for signal 6 didn't terminate the process? Feb 04 09:32:42 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Main process exited, code=exited, status=1/FAILURE Feb 04 09:32:42 osd07 bash[31529]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:42 osd07 bash[31529]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-deactivate Feb 04 09:32:42 osd07 bash[31529]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:43 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Failed with result 'exit-code'. Feb 04 09:32:53 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Service RestartSec=10s expired, scheduling restart. Feb 04 09:32:53 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Scheduled restart job, restart counter is at 3. Feb 04 09:32:53 osd07 systemd[1]: Stopped Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:53 osd07 systemd[1]: Starting Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e... Feb 04 09:32:53 osd07 docker[31660]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:53 osd07 docker[31660]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:53 osd07 systemd[1]: Started Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:32:53 osd07 bash[31674]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:53 osd07 bash[31674]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-activate Feb 04 09:32:53 osd07 bash[31674]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:54 osd07 bash[31674]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:54 osd07 bash[31674]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 --path /var/lib/ceph/osd/ceph-18 --no-mon-config Feb 04 09:32:54 osd07 bash[31674]: Running command: /usr/bin/ln -snf /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:54 osd07 bash[31674]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:54 osd07 bash[31674]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7 Feb 04 09:32:54 osd07 bash[31674]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:32:54 osd07 bash[31674]: --> ceph-volume lvm activate successful for osd ID: 18 Feb 04 09:32:54 osd07 bash[31674]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:54 osd07 bash[31674]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:32:54 osd07 bash[31674]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f34705fe700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f346fdfd700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f346f5fc700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev aio_wait 0x556cab128bb0 done Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.509+0000 7f346fdfd700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.549+0000 7f346f5fc700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.549+0000 7f346f5fc700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.549+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.761+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f34705fe700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f34705fe700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f347908af40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.805+0000 7f347908af40 -1 Falling back to public interface Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 0 load: jerasure load: lrc load: isa Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f346f5fc700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:55 osd07 bash[31674]: debug 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.069+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f346f5fc700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f346f5fc700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f346c585700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f346bd84700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f346b583700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluefs mount Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluefs _open_super Feb 04 09:32:56 osd07 bash[31674]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f347908af40 time 2021-02-04T08:32:56.104984+0000 Feb 04 09:32:56 osd07 bash[31674]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f347908af40 5 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.101+0000 7f346ad82700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x556c9f6e2372] Feb 04 09:32:56 osd07 bash[31674]: 2: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 4: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 5: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 10: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 11: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 12: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 13: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: *** Caught signal (Aborted) ** Feb 04 09:32:56 osd07 bash[31674]: in thread 7f347908af40 thread_name:ceph-osd Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.105+0000 7f347908af40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f347908af40 time 2021-02-04T08:32:56.104984+0000 Feb 04 09:32:56 osd07 bash[31674]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x556c9f6e2372] Feb 04 09:32:56 osd07 bash[31674]: 2: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 4: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 5: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 10: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 11: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 12: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 13: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (()+0x12b20) [0x7f3476df2b20] Feb 04 09:32:56 osd07 bash[31674]: 2: (gsignal()+0x10f) [0x7f3475a5a7ff] Feb 04 09:32:56 osd07 bash[31674]: 3: (abort()+0x127) [0x7f3475a44c35] Feb 04 09:32:56 osd07 bash[31674]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x556c9f6e23c3] Feb 04 09:32:56 osd07 bash[31674]: 5: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 13: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 14: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 15: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 16: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: debug 2021-02-04T08:32:56.109+0000 7f347908af40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:56 osd07 bash[31674]: in thread 7f347908af40 thread_name:ceph-osd Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (()+0x12b20) [0x7f3476df2b20] Feb 04 09:32:56 osd07 bash[31674]: 2: (gsignal()+0x10f) [0x7f3475a5a7ff] Feb 04 09:32:56 osd07 bash[31674]: 3: (abort()+0x127) [0x7f3475a44c35] Feb 04 09:32:56 osd07 bash[31674]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x556c9f6e23c3] Feb 04 09:32:56 osd07 bash[31674]: 5: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 13: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 14: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 15: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 16: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:56 osd07 bash[31674]: --- begin dump of recent events --- Feb 04 09:32:56 osd07 bash[31674]: debug -376> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command assert hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -375> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command abort hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -374> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command leak_some_memory hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -373> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perfcounters_dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -372> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command 1 hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -371> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -370> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perfcounters_schema hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -369> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf histogram dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -368> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command 2 hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -367> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf schema hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -366> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf histogram schema hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -365> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf reset hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -364> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config show hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -363> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config help hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -362> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config set hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -361> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config unset hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -360> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config get hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -359> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config diff hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -358> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config diff get hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -357> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command injectargs hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -356> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command log flush hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -355> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command log dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -354> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command log reopen hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -353> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command dump_mempools hook 0x556cabd74068 Feb 04 09:32:56 osd07 bash[31674]: debug -352> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: get_monmap_and_config Feb 04 09:32:56 osd07 bash[31674]: debug -351> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: build_initial_monmap Feb 04 09:32:56 osd07 bash[31674]: debug -350> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: monmap: Feb 04 09:32:56 osd07 bash[31674]: epoch 0 Feb 04 09:32:56 osd07 bash[31674]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:56 osd07 bash[31674]: last_changed 2021-02-04T08:32:55.256166+0000 Feb 04 09:32:56 osd07 bash[31674]: created 2021-02-04T08:32:55.256166+0000 Feb 04 09:32:56 osd07 bash[31674]: min_mon_release 0 (unknown) Feb 04 09:32:56 osd07 bash[31674]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:56 osd07 bash[31674]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:56 osd07 bash[31674]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:56 osd07 bash[31674]: debug -349> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -348> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -347> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -346> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -345> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -344> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -343> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -342> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -341> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -340> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -339> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -338> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -337> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -336> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -335> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -334> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -333> 2021-02-04T08:32:55.253+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -332> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: init Feb 04 09:32:56 osd07 bash[31674]: debug -331> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -330> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -329> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -328> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -327> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -326> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -325> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -324> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -323> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -322> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -321> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -320> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -319> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -318> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -317> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -316> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -315> 2021-02-04T08:32:55.253+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -314> 2021-02-04T08:32:55.253+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -313> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: _reopen_session rank -1 Feb 04 09:32:56 osd07 bash[31674]: debug -312> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: _add_conns ranks=[2,1,0] Feb 04 09:32:56 osd07 bash[31674]: debug -311> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): picked mon.noname-c con 0x556cab19ec00 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:56 osd07 bash[31674]: debug -310> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): picked mon.noname-b con 0x556cab19f000 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:56 osd07 bash[31674]: debug -309> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): picked mon.noname-a con 0x556cab19f400 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:56 osd07 bash[31674]: debug -308> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): start opening mon connection Feb 04 09:32:56 osd07 bash[31674]: debug -307> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): start opening mon connection Feb 04 09:32:56 osd07 bash[31674]: debug -306> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): start opening mon connection Feb 04 09:32:56 osd07 bash[31674]: debug -305> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): _renew_subs Feb 04 09:32:56 osd07 bash[31674]: debug -304> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:55.257366+0000 Feb 04 09:32:56 osd07 bash[31674]: debug -303> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): get_auth_request con 0x556cab19f400 auth_method 0 Feb 04 09:32:56 osd07 bash[31674]: debug -302> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:56 osd07 bash[31674]: debug -301> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:56 osd07 bash[31674]: debug -300> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): get_auth_request con 0x556cab19f000 auth_method 0 Feb 04 09:32:56 osd07 bash[31674]: debug -299> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:56 osd07 bash[31674]: debug -298> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:56 osd07 bash[31674]: debug -297> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): get_auth_request con 0x556cab19ec00 auth_method 0 Feb 04 09:32:56 osd07 bash[31674]: debug -296> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:56 osd07 bash[31674]: debug -295> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:56 osd07 bash[31674]: debug -294> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:56 osd07 bash[31674]: debug -293> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:56 osd07 bash[31674]: debug -292> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:56 osd07 bash[31674]: debug -291> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:56 osd07 bash[31674]: debug -290> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:56 osd07 bash[31674]: debug -289> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:56 osd07 bash[31674]: debug -288> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:56 osd07 bash[31674]: debug -287> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:56 osd07 bash[31674]: debug -286> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:56 osd07 bash[31674]: debug -285> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_done global_id 3642732 payload 450 Feb 04 09:32:56 osd07 bash[31674]: debug -284> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient: _finish_hunting 0 Feb 04 09:32:56 osd07 bash[31674]: debug -283> 2021-02-04T08:32:55.257+0000 7f3473604700 1 monclient: found mon.noname-a Feb 04 09:32:56 osd07 bash[31674]: debug -282> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient: _send_mon_message to mon.noname-a at v2:10.10.0.14:3300/0 Feb 04 09:32:56 osd07 bash[31674]: debug -281> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:56 osd07 bash[31674]: debug -280> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: got monmap 9 from mon.noname-a (according to old e9) Feb 04 09:32:56 osd07 bash[31674]: debug -279> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: dump: Feb 04 09:32:56 osd07 bash[31674]: epoch 9 Feb 04 09:32:56 osd07 bash[31674]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:56 osd07 bash[31674]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:56 osd07 bash[31674]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:56 osd07 bash[31674]: min_mon_release 15 (octopus) Feb 04 09:32:56 osd07 bash[31674]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:56 osd07 bash[31674]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:56 osd07 bash[31674]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:56 osd07 bash[31674]: debug -278> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: _finish_auth 0 Feb 04 09:32:56 osd07 bash[31674]: debug -277> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:32:25.259377+0000) Feb 04 09:32:56 osd07 bash[31674]: debug -276> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: _send_mon_message to mon.osd04 at v2:10.10.0.14:3300/0 Feb 04 09:32:56 osd07 bash[31674]: debug -275> 2021-02-04T08:32:55.257+0000 7f347908af40 5 monclient: authenticate success, global_id 3642732 Feb 04 09:32:56 osd07 bash[31674]: debug -274> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:56 osd07 bash[31674]: debug -273> 2021-02-04T08:32:55.257+0000 7f347908af40 10 monclient: get_monmap_and_config success Feb 04 09:32:56 osd07 bash[31674]: debug -272> 2021-02-04T08:32:55.257+0000 7f347908af40 10 monclient: shutdown Feb 04 09:32:56 osd07 bash[31674]: debug -271> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 Feb 04 09:32:56 osd07 bash[31674]: debug -270> 2021-02-04T08:32:55.257+0000 7f34705fe700 4 set_mon_vals no callback set Feb 04 09:32:56 osd07 bash[31674]: debug -269> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:56 osd07 bash[31674]: debug -268> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:56 osd07 bash[31674]: debug -267> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:56 osd07 bash[31674]: debug -266> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:56 osd07 bash[31674]: debug -265> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:56 osd07 bash[31674]: debug -264> 2021-02-04T08:32:55.257+0000 7f347908af40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:56 osd07 bash[31674]: debug -263> 2021-02-04T08:32:55.257+0000 7f347908af40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:56 osd07 bash[31674]: debug -262> 2021-02-04T08:32:55.257+0000 7f347908af40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:56 osd07 bash[31674]: debug -261> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:56 osd07 bash[31674]: debug -260> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:56 osd07 bash[31674]: debug -259> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command 0 hook 0x556cab0f8388 Feb 04 09:32:56 osd07 bash[31674]: debug -258> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command version hook 0x556cab0f8388 Feb 04 09:32:56 osd07 bash[31674]: debug -257> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command git_version hook 0x556cab0f8388 Feb 04 09:32:56 osd07 bash[31674]: debug -256> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command help hook 0x556cab0f44c0 Feb 04 09:32:56 osd07 bash[31674]: debug -255> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command get_command_descriptions hook 0x556cab0f4070 Feb 04 09:32:56 osd07 bash[31674]: debug -254> 2021-02-04T08:32:55.257+0000 7f347908af40 5 object store type is bluestore Feb 04 09:32:56 osd07 bash[31674]: debug -253> 2021-02-04T08:32:55.257+0000 7f3471600700 5 asok(0x556cab1a0000) entry start Feb 04 09:32:56 osd07 bash[31674]: debug -252> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:56 osd07 bash[31674]: debug -251> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -250> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -249> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -248> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -247> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -246> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -245> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -244> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -243> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -242> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -241> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -240> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -239> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:56 osd07 bash[31674]: debug -238> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -237> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -236> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -235> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -234> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -233> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -232> 2021-02-04T08:32:55.261+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -231> 2021-02-04T08:32:55.261+0000 7f34705fe700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -230> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -229> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug -228> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluestore bluefs available hook 0x556cab0f40b0 Feb 04 09:32:56 osd07 bash[31674]: debug -227> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluefs stats hook 0x556cab0f40b0 Feb 04 09:32:56 osd07 bash[31674]: debug -226> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -225> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -224> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -223> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -222> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -221> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -220> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -219> 2021-02-04T08:32:55.261+0000 7f346fdfd700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -218> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -217> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:56 osd07 bash[31674]: debug -216> 2021-02-04T08:32:55.261+0000 7f346f5fc700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -215> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) unregister_commands bluefs stats Feb 04 09:32:56 osd07 bash[31674]: debug -214> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) unregister_commands bluestore bluefs available Feb 04 09:32:56 osd07 bash[31674]: debug -213> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev aio_wait 0x556cab128bb0 done Feb 04 09:32:56 osd07 bash[31674]: debug -212> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:56 osd07 bash[31674]: debug -211> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug -210> 2021-02-04T08:32:55.509+0000 7f346fdfd700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug -209> 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug -208> 2021-02-04T08:32:55.549+0000 7f346f5fc700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug -207> 2021-02-04T08:32:55.549+0000 7f346f5fc700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug -206> 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug -205> 2021-02-04T08:32:55.549+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:56 osd07 bash[31674]: debug -204> 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug -203> 2021-02-04T08:32:55.761+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug -202> 2021-02-04T08:32:55.805+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug -201> 2021-02-04T08:32:55.805+0000 7f34705fe700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug -200> 2021-02-04T08:32:55.805+0000 7f34705fe700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug -199> 2021-02-04T08:32:55.805+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug -198> 2021-02-04T08:32:55.805+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:56 osd07 bash[31674]: debug -197> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -196> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -195> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -194> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -193> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -192> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -191> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -190> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -189> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -188> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -187> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -186> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -185> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -184> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -183> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -182> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -181> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -180> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -179> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -178> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -177> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -176> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -175> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -174> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -173> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -172> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -171> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -170> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -169> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -168> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -167> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -166> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -165> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -164> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -163> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -162> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -161> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -160> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -159> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -158> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -157> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -156> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -155> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -154> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -153> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -152> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -151> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -150> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -149> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -148> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -147> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -146> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -145> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -144> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -143> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -142> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -141> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -140> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -139> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -138> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -137> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -136> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -135> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -134> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -133> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -132> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -131> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -130> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -129> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -128> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -127> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -126> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -125> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -124> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -123> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -122> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -121> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -120> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -119> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -118> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -117> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -116> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -115> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -114> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -113> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -112> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -111> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -110> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -109> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -108> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -107> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -106> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -105> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -104> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -103> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -102> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -101> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -100> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -99> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -98> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -97> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -96> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -95> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -94> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -93> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -92> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -91> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -90> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -89> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -88> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -87> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -86> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -85> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -84> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -83> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -82> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -81> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -80> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -79> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -78> 2021-02-04T08:32:55.805+0000 7f347908af40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:56 osd07 bash[31674]: debug -77> 2021-02-04T08:32:55.805+0000 7f347908af40 -1 Falling back to public interface Feb 04 09:32:56 osd07 bash[31674]: debug -76> 2021-02-04T08:32:55.809+0000 7f347908af40 10 monclient: build_initial_monmap Feb 04 09:32:56 osd07 bash[31674]: debug -75> 2021-02-04T08:32:55.809+0000 7f347908af40 10 monclient: monmap: Feb 04 09:32:56 osd07 bash[31674]: epoch 0 Feb 04 09:32:56 osd07 bash[31674]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:56 osd07 bash[31674]: last_changed 0.000000 Feb 04 09:32:56 osd07 bash[31674]: created 0.000000 Feb 04 09:32:56 osd07 bash[31674]: min_mon_release 0 (unknown) Feb 04 09:32:56 osd07 bash[31674]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:56 osd07 bash[31674]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:56 osd07 bash[31674]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:56 osd07 bash[31674]: debug -74> 2021-02-04T08:32:55.817+0000 7f347908af40 0 load: jerasure load: lrc load: isa Feb 04 09:32:56 osd07 bash[31674]: debug -73> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -72> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -71> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:56 osd07 bash[31674]: debug -70> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -69> 2021-02-04T08:32:55.817+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -68> 2021-02-04T08:32:55.817+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -67> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -66> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -65> 2021-02-04T08:32:55.817+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -64> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -63> 2021-02-04T08:32:55.817+0000 7f346f5fc700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -62> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -61> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug -60> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:56 osd07 bash[31674]: debug -59> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug -58> 2021-02-04T08:32:56.069+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug -57> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug -56> 2021-02-04T08:32:56.101+0000 7f346f5fc700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug -55> 2021-02-04T08:32:56.101+0000 7f346f5fc700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug -54> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug -53> 2021-02-04T08:32:56.101+0000 7f347908af40 5 asok(0x556cab1a0000) register_command objecter_requests hook 0x556cab0f4960 Feb 04 09:32:56 osd07 bash[31674]: debug -52> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -51> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -50> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -49> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -48> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -47> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -46> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -45> 2021-02-04T08:32:56.101+0000 7f347908af40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:56 osd07 bash[31674]: debug -44> 2021-02-04T08:32:56.101+0000 7f347908af40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:56 osd07 bash[31674]: debug -43> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:56 osd07 bash[31674]: debug -42> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:56 osd07 bash[31674]: debug -41> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -40> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -39> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -38> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -37> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:56 osd07 bash[31674]: debug -36> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -35> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -34> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -33> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -32> 2021-02-04T08:32:56.101+0000 7f346c585700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -31> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -30> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -29> 2021-02-04T08:32:56.101+0000 7f346bd84700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -28> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -27> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug -26> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -25> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -24> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -23> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -22> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:56 osd07 bash[31674]: debug -21> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -20> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -19> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:56 osd07 bash[31674]: debug -18> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:56 osd07 bash[31674]: debug -17> 2021-02-04T08:32:56.101+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluestore bluefs available hook 0x556cab0f4970 Feb 04 09:32:56 osd07 bash[31674]: debug -16> 2021-02-04T08:32:56.101+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluefs stats hook 0x556cab0f4970 Feb 04 09:32:56 osd07 bash[31674]: debug -15> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -14> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -13> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -12> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -11> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -10> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -9> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -8> 2021-02-04T08:32:56.101+0000 7f346b583700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -7> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -6> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:56 osd07 bash[31674]: debug -5> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluefs mount Feb 04 09:32:56 osd07 bash[31674]: debug -4> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluefs _open_super Feb 04 09:32:56 osd07 bash[31674]: debug -3> 2021-02-04T08:32:56.101+0000 7f347908af40 5 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:56 osd07 bash[31674]: debug -2> 2021-02-04T08:32:56.101+0000 7f346ad82700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -1> 2021-02-04T08:32:56.105+0000 7f347908af40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f347908af40 time 2021-02-04T08:32:56.104984+0000 Feb 04 09:32:56 osd07 bash[31674]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x556c9f6e2372] Feb 04 09:32:56 osd07 bash[31674]: 2: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 4: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 5: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 10: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 11: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 12: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 13: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: debug 0> 2021-02-04T08:32:56.109+0000 7f347908af40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:56 osd07 bash[31674]: in thread 7f347908af40 thread_name:ceph-osd Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (()+0x12b20) [0x7f3476df2b20] Feb 04 09:32:56 osd07 bash[31674]: 2: (gsignal()+0x10f) [0x7f3475a5a7ff] Feb 04 09:32:56 osd07 bash[31674]: 3: (abort()+0x127) [0x7f3475a44c35] Feb 04 09:32:56 osd07 bash[31674]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x556c9f6e23c3] Feb 04 09:32:56 osd07 bash[31674]: 5: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 13: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 14: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 15: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 16: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:56 osd07 bash[31674]: --- logging levels --- Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 none Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 lockdep Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 context Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 crush Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_balancer Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_locker Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_log Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_log_expire Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_migrator Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 buffer Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 timer Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 filer Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 striper Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 objecter Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rados Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd_mirror Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd_replay Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd_rwl Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 journaler Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 objectcacher Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 immutable_obj_cache Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 client Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 osd Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 optracker Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 objclass Feb 04 09:32:56 osd07 bash[31674]: 1/ 3 filestore Feb 04 09:32:56 osd07 bash[31674]: 1/ 3 journal Feb 04 09:32:56 osd07 bash[31674]: 0/ 0 ms Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mon Feb 04 09:32:56 osd07 bash[31674]: 0/10 monc Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 paxos Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 tp Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 auth Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 crypto Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 finisher Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 reserver Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 heartbeatmap Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 perfcounter Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 rgw Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 rgw_sync Feb 04 09:32:56 osd07 bash[31674]: 1/10 civetweb Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 javaclient Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 asok Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 throttle Feb 04 09:32:56 osd07 bash[31674]: 0/ 0 refs Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 compressor Feb 04 09:32:56 osd07 bash[31674]: 20/20 bluestore Feb 04 09:32:56 osd07 bash[31674]: 20/20 bluefs Feb 04 09:32:56 osd07 bash[31674]: 20/20 bdev Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 kstore Feb 04 09:32:56 osd07 bash[31674]: 4/ 5 rocksdb Feb 04 09:32:56 osd07 bash[31674]: 4/ 5 leveldb Feb 04 09:32:56 osd07 bash[31674]: 4/ 5 memdb Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 fuse Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mgr Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mgrc Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 dpdk Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 eventtrace Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 prioritycache Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 test Feb 04 09:32:56 osd07 bash[31674]: -2/-2 (syslog threshold) Feb 04 09:32:56 osd07 bash[31674]: 99/99 (stderr threshold) Feb 04 09:32:56 osd07 bash[31674]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:56 osd07 bash[31674]: 7f346ad82700 / bstore_discard Feb 04 09:32:56 osd07 bash[31674]: 7f346b583700 / bstore_aio Feb 04 09:32:56 osd07 bash[31674]: 7f346bd84700 / bstore_discard Feb 04 09:32:56 osd07 bash[31674]: 7f346c585700 / bstore_aio Feb 04 09:32:56 osd07 bash[31674]: 7f346f5fc700 / OpHistorySvc Feb 04 09:32:56 osd07 bash[31674]: 7f346fdfd700 / safe_timer Feb 04 09:32:56 osd07 bash[31674]: 7f34705fe700 / signal_handler Feb 04 09:32:56 osd07 bash[31674]: 7f3470dff700 / ceph-osd Feb 04 09:32:56 osd07 bash[31674]: 7f3471600700 / admin_socket Feb 04 09:32:56 osd07 bash[31674]: 7f3471e01700 / service Feb 04 09:32:56 osd07 bash[31674]: 7f3472602700 / msgr-worker-2 Feb 04 09:32:56 osd07 bash[31674]: 7f3472e03700 / msgr-worker-1 Feb 04 09:32:56 osd07 bash[31674]: 7f3473604700 / msgr-worker-0 Feb 04 09:32:56 osd07 bash[31674]: 7f347908af40 / ceph-osd Feb 04 09:32:56 osd07 bash[31674]: max_recent 10000 Feb 04 09:32:56 osd07 bash[31674]: max_new 1000 Feb 04 09:32:56 osd07 bash[31674]: log_file Feb 04 09:32:56 osd07 bash[31674]: --- end dump of recent events --- Feb 04 09:32:56 osd07 bash[31674]: --- begin dump of recent events --- Feb 04 09:32:56 osd07 bash[31674]: debug -376> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command assert hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -375> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command abort hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -374> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command leak_some_memory hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -373> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perfcounters_dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -372> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command 1 hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -371> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -370> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perfcounters_schema hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -369> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf histogram dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -368> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command 2 hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -367> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf schema hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -366> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf histogram schema hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -365> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command perf reset hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -364> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config show hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -363> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config help hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -362> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config set hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -361> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config unset hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -360> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config get hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -359> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config diff hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -358> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command config diff get hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -357> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command injectargs hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -356> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command log flush hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -355> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command log dump hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -354> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command log reopen hook 0x556cab0f45b0 Feb 04 09:32:56 osd07 bash[31674]: debug -353> 2021-02-04T08:32:55.249+0000 7f347908af40 5 asok(0x556cab1a0000) register_command dump_mempools hook 0x556cabd74068 Feb 04 09:32:56 osd07 bash[31674]: debug -352> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: get_monmap_and_config Feb 04 09:32:56 osd07 bash[31674]: debug -351> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: build_initial_monmap Feb 04 09:32:56 osd07 bash[31674]: debug -350> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: monmap: Feb 04 09:32:56 osd07 bash[31674]: epoch 0 Feb 04 09:32:56 osd07 bash[31674]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:56 osd07 bash[31674]: last_changed 2021-02-04T08:32:55.256166+0000 Feb 04 09:32:56 osd07 bash[31674]: created 2021-02-04T08:32:55.256166+0000 Feb 04 09:32:56 osd07 bash[31674]: min_mon_release 0 (unknown) Feb 04 09:32:56 osd07 bash[31674]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:56 osd07 bash[31674]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:56 osd07 bash[31674]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:56 osd07 bash[31674]: debug -349> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -348> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -347> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -346> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -345> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -344> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -343> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -342> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -341> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -340> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -339> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -338> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -337> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -336> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -335> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -334> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x556cab18a940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -333> 2021-02-04T08:32:55.253+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -332> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: init Feb 04 09:32:56 osd07 bash[31674]: debug -331> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -330> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -329> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -328> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -327> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -326> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -325> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -324> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -323> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -322> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -321> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -320> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -319> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -318> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -317> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -316> 2021-02-04T08:32:55.253+0000 7f347908af40 5 AuthRegistry(0x7ffe738f6950) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -315> 2021-02-04T08:32:55.253+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -314> 2021-02-04T08:32:55.253+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -313> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: _reopen_session rank -1 Feb 04 09:32:56 osd07 bash[31674]: debug -312> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient: _add_conns ranks=[2,1,0] Feb 04 09:32:56 osd07 bash[31674]: debug -311> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): picked mon.noname-c con 0x556cab19ec00 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:32:56 osd07 bash[31674]: debug -310> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): picked mon.noname-b con 0x556cab19f000 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:32:56 osd07 bash[31674]: debug -309> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): picked mon.noname-a con 0x556cab19f400 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:32:56 osd07 bash[31674]: debug -308> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): start opening mon connection Feb 04 09:32:56 osd07 bash[31674]: debug -307> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): start opening mon connection Feb 04 09:32:56 osd07 bash[31674]: debug -306> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): start opening mon connection Feb 04 09:32:56 osd07 bash[31674]: debug -305> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): _renew_subs Feb 04 09:32:56 osd07 bash[31674]: debug -304> 2021-02-04T08:32:55.253+0000 7f347908af40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:37:55.257366+0000 Feb 04 09:32:56 osd07 bash[31674]: debug -303> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): get_auth_request con 0x556cab19f400 auth_method 0 Feb 04 09:32:56 osd07 bash[31674]: debug -302> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:56 osd07 bash[31674]: debug -301> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:56 osd07 bash[31674]: debug -300> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): get_auth_request con 0x556cab19f000 auth_method 0 Feb 04 09:32:56 osd07 bash[31674]: debug -299> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:56 osd07 bash[31674]: debug -298> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:56 osd07 bash[31674]: debug -297> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): get_auth_request con 0x556cab19ec00 auth_method 0 Feb 04 09:32:56 osd07 bash[31674]: debug -296> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:32:56 osd07 bash[31674]: debug -295> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): _init_auth method 2 Feb 04 09:32:56 osd07 bash[31674]: debug -294> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:56 osd07 bash[31674]: debug -293> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:56 osd07 bash[31674]: debug -292> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:56 osd07 bash[31674]: debug -291> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:56 osd07 bash[31674]: debug -290> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:56 osd07 bash[31674]: debug -289> 2021-02-04T08:32:55.257+0000 7f3472602700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:56 osd07 bash[31674]: debug -288> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:32:56 osd07 bash[31674]: debug -287> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:32:56 osd07 bash[31674]: debug -286> 2021-02-04T08:32:55.257+0000 7f3472e03700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:32:56 osd07 bash[31674]: debug -285> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient(hunting): handle_auth_done global_id 3642732 payload 450 Feb 04 09:32:56 osd07 bash[31674]: debug -284> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient: _finish_hunting 0 Feb 04 09:32:56 osd07 bash[31674]: debug -283> 2021-02-04T08:32:55.257+0000 7f3473604700 1 monclient: found mon.noname-a Feb 04 09:32:56 osd07 bash[31674]: debug -282> 2021-02-04T08:32:55.257+0000 7f3473604700 10 monclient: _send_mon_message to mon.noname-a at v2:10.10.0.14:3300/0 Feb 04 09:32:56 osd07 bash[31674]: debug -281> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:32:56 osd07 bash[31674]: debug -280> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: got monmap 9 from mon.noname-a (according to old e9) Feb 04 09:32:56 osd07 bash[31674]: debug -279> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: dump: Feb 04 09:32:56 osd07 bash[31674]: epoch 9 Feb 04 09:32:56 osd07 bash[31674]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:32:56 osd07 bash[31674]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:32:56 osd07 bash[31674]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:32:56 osd07 bash[31674]: min_mon_release 15 (octopus) Feb 04 09:32:56 osd07 bash[31674]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:32:56 osd07 bash[31674]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:32:56 osd07 bash[31674]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:32:56 osd07 bash[31674]: debug -278> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: _finish_auth 0 Feb 04 09:32:56 osd07 bash[31674]: debug -277> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:32:25.259377+0000) Feb 04 09:32:56 osd07 bash[31674]: debug -276> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: _send_mon_message to mon.osd04 at v2:10.10.0.14:3300/0 Feb 04 09:32:56 osd07 bash[31674]: debug -275> 2021-02-04T08:32:55.257+0000 7f347908af40 5 monclient: authenticate success, global_id 3642732 Feb 04 09:32:56 osd07 bash[31674]: debug -274> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:32:56 osd07 bash[31674]: debug -273> 2021-02-04T08:32:55.257+0000 7f347908af40 10 monclient: get_monmap_and_config success Feb 04 09:32:56 osd07 bash[31674]: debug -272> 2021-02-04T08:32:55.257+0000 7f347908af40 10 monclient: shutdown Feb 04 09:32:56 osd07 bash[31674]: debug -271> 2021-02-04T08:32:55.257+0000 7f3471e01700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 Feb 04 09:32:56 osd07 bash[31674]: debug -270> 2021-02-04T08:32:55.257+0000 7f34705fe700 4 set_mon_vals no callback set Feb 04 09:32:56 osd07 bash[31674]: debug -269> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:32:56 osd07 bash[31674]: debug -268> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:32:56 osd07 bash[31674]: debug -267> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:32:56 osd07 bash[31674]: debug -266> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:32:56 osd07 bash[31674]: debug -265> 2021-02-04T08:32:55.257+0000 7f34705fe700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:32:56 osd07 bash[31674]: debug -264> 2021-02-04T08:32:55.257+0000 7f347908af40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:32:56 osd07 bash[31674]: debug -263> 2021-02-04T08:32:55.257+0000 7f347908af40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:32:56 osd07 bash[31674]: debug -262> 2021-02-04T08:32:55.257+0000 7f347908af40 0 pidfile_write: ignore empty --pid-file Feb 04 09:32:56 osd07 bash[31674]: debug -261> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:56 osd07 bash[31674]: debug -260> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:32:56 osd07 bash[31674]: debug -259> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command 0 hook 0x556cab0f8388 Feb 04 09:32:56 osd07 bash[31674]: debug -258> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command version hook 0x556cab0f8388 Feb 04 09:32:56 osd07 bash[31674]: debug -257> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command git_version hook 0x556cab0f8388 Feb 04 09:32:56 osd07 bash[31674]: debug -256> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command help hook 0x556cab0f44c0 Feb 04 09:32:56 osd07 bash[31674]: debug -255> 2021-02-04T08:32:55.257+0000 7f347908af40 5 asok(0x556cab1a0000) register_command get_command_descriptions hook 0x556cab0f4070 Feb 04 09:32:56 osd07 bash[31674]: debug -254> 2021-02-04T08:32:55.257+0000 7f347908af40 5 object store type is bluestore Feb 04 09:32:56 osd07 bash[31674]: debug -253> 2021-02-04T08:32:55.257+0000 7f3471600700 5 asok(0x556cab1a0000) entry start Feb 04 09:32:56 osd07 bash[31674]: debug -252> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:32:56 osd07 bash[31674]: debug -251> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -250> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -249> 2021-02-04T08:32:55.257+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -248> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -247> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -246> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -245> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -244> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -243> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -242> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -241> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -240> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -239> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:32:56 osd07 bash[31674]: debug -238> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -237> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -236> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -235> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -234> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -233> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -232> 2021-02-04T08:32:55.261+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -231> 2021-02-04T08:32:55.261+0000 7f34705fe700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -230> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -229> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug -228> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluestore bluefs available hook 0x556cab0f40b0 Feb 04 09:32:56 osd07 bash[31674]: debug -227> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluefs stats hook 0x556cab0f40b0 Feb 04 09:32:56 osd07 bash[31674]: debug -226> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -225> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -224> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -223> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -222> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -221> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -220> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -219> 2021-02-04T08:32:55.261+0000 7f346fdfd700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -218> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -217> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:56 osd07 bash[31674]: debug -216> 2021-02-04T08:32:55.261+0000 7f346f5fc700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -215> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) unregister_commands bluefs stats Feb 04 09:32:56 osd07 bash[31674]: debug -214> 2021-02-04T08:32:55.261+0000 7f347908af40 5 asok(0x556cab1a0000) unregister_commands bluestore bluefs available Feb 04 09:32:56 osd07 bash[31674]: debug -213> 2021-02-04T08:32:55.261+0000 7f347908af40 20 bdev aio_wait 0x556cab128bb0 done Feb 04 09:32:56 osd07 bash[31674]: debug -212> 2021-02-04T08:32:55.261+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:56 osd07 bash[31674]: debug -211> 2021-02-04T08:32:55.261+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug -210> 2021-02-04T08:32:55.509+0000 7f346fdfd700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug -209> 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug -208> 2021-02-04T08:32:55.549+0000 7f346f5fc700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug -207> 2021-02-04T08:32:55.549+0000 7f346f5fc700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug -206> 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug -205> 2021-02-04T08:32:55.549+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:56 osd07 bash[31674]: debug -204> 2021-02-04T08:32:55.549+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug -203> 2021-02-04T08:32:55.761+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug -202> 2021-02-04T08:32:55.805+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug -201> 2021-02-04T08:32:55.805+0000 7f34705fe700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug -200> 2021-02-04T08:32:55.805+0000 7f34705fe700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug -199> 2021-02-04T08:32:55.805+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug -198> 2021-02-04T08:32:55.805+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:32:56 osd07 bash[31674]: debug -197> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -196> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -195> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -194> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -193> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -192> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -191> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -190> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -189> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -188> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -187> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -186> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -185> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -184> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -183> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -182> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18b940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -181> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -180> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -179> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -178> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -177> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -176> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -175> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -174> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -173> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -172> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -171> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -170> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -169> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -168> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -167> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -166> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -165> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18c940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -164> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -163> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -162> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -161> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -160> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -159> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -158> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -157> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -156> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -155> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -154> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -153> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -152> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -151> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -150> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -149> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -148> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cab18d940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -147> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -146> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -145> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -144> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -143> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -142> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -141> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -140> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -139> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -138> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -137> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -136> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -135> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -134> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -133> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -132> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -131> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4e940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -130> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -129> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -128> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -127> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -126> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -125> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -124> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -123> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -122> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -121> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -120> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -119> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -118> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -117> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -116> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -115> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -114> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe4f940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -113> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -112> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -111> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -110> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -109> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -108> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -107> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -106> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -105> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -104> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -103> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -102> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -101> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -100> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -99> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -98> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -97> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe50940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -96> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -95> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -94> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -93> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: cephx Feb 04 09:32:56 osd07 bash[31674]: debug -92> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding auth protocol: none Feb 04 09:32:56 osd07 bash[31674]: debug -91> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -90> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -89> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -88> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -87> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -86> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -85> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -84> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -83> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -82> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -81> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: crc Feb 04 09:32:56 osd07 bash[31674]: debug -80> 2021-02-04T08:32:55.805+0000 7f347908af40 5 AuthRegistry(0x556cabe51940) adding con mode: secure Feb 04 09:32:56 osd07 bash[31674]: debug -79> 2021-02-04T08:32:55.805+0000 7f347908af40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:32:56 osd07 bash[31674]: debug -78> 2021-02-04T08:32:55.805+0000 7f347908af40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:56 osd07 bash[31674]: debug -77> 2021-02-04T08:32:55.805+0000 7f347908af40 -1 Falling back to public interface Feb 04 09:32:56 osd07 bash[31674]: debug -76> 2021-02-04T08:32:55.809+0000 7f347908af40 10 monclient: build_initial_monmap Feb 04 09:32:56 osd07 bash[31674]: debug -75> 2021-02-04T08:32:55.809+0000 7f347908af40 10 monclient: monmap: Feb 04 09:32:56 osd07 bash[31674]: epoch 0 Feb 04 09:32:56 osd07 bash[31674]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:32:56 osd07 bash[31674]: last_changed 0.000000 Feb 04 09:32:56 osd07 bash[31674]: created 0.000000 Feb 04 09:32:56 osd07 bash[31674]: min_mon_release 0 (unknown) Feb 04 09:32:56 osd07 bash[31674]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:32:56 osd07 bash[31674]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:32:56 osd07 bash[31674]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:32:56 osd07 bash[31674]: debug -74> 2021-02-04T08:32:55.817+0000 7f347908af40 0 load: jerasure load: lrc load: isa Feb 04 09:32:56 osd07 bash[31674]: debug -73> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -72> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -71> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:56 osd07 bash[31674]: debug -70> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -69> 2021-02-04T08:32:55.817+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -68> 2021-02-04T08:32:55.817+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -67> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -66> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -65> 2021-02-04T08:32:55.817+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -64> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -63> 2021-02-04T08:32:55.817+0000 7f346f5fc700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -62> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -61> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug -60> 2021-02-04T08:32:55.817+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:32:56 osd07 bash[31674]: debug -59> 2021-02-04T08:32:55.817+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:32:56 osd07 bash[31674]: debug -58> 2021-02-04T08:32:56.069+0000 7f3470dff700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:32:56 osd07 bash[31674]: debug -57> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:32:56 osd07 bash[31674]: debug -56> 2021-02-04T08:32:56.101+0000 7f346f5fc700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:32:56 osd07 bash[31674]: debug -55> 2021-02-04T08:32:56.101+0000 7f346f5fc700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:32:56 osd07 bash[31674]: debug -54> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:32:56 osd07 bash[31674]: debug -53> 2021-02-04T08:32:56.101+0000 7f347908af40 5 asok(0x556cab1a0000) register_command objecter_requests hook 0x556cab0f4960 Feb 04 09:32:56 osd07 bash[31674]: debug -52> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -51> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -50> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -49> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -48> 2021-02-04T08:32:56.101+0000 7f347908af40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:32:56 osd07 bash[31674]: debug -47> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -46> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -45> 2021-02-04T08:32:56.101+0000 7f347908af40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:32:56 osd07 bash[31674]: debug -44> 2021-02-04T08:32:56.101+0000 7f347908af40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:32:56 osd07 bash[31674]: debug -43> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:32:56 osd07 bash[31674]: debug -42> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:32:56 osd07 bash[31674]: debug -41> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -40> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -39> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -38> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -37> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:32:56 osd07 bash[31674]: debug -36> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -35> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -34> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -33> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -32> 2021-02-04T08:32:56.101+0000 7f346c585700 10 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -31> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -30> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -29> 2021-02-04T08:32:56.101+0000 7f346bd84700 20 bdev(0x556cabe2a000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -28> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -27> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:32:56 osd07 bash[31674]: debug -26> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -25> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -24> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -23> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -22> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:32:56 osd07 bash[31674]: debug -21> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:32:56 osd07 bash[31674]: debug -20> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:32:56 osd07 bash[31674]: debug -19> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:32:56 osd07 bash[31674]: debug -18> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:32:56 osd07 bash[31674]: debug -17> 2021-02-04T08:32:56.101+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluestore bluefs available hook 0x556cab0f4970 Feb 04 09:32:56 osd07 bash[31674]: debug -16> 2021-02-04T08:32:56.101+0000 7f347908af40 5 asok(0x556cab1a0000) register_command bluefs stats hook 0x556cab0f4970 Feb 04 09:32:56 osd07 bash[31674]: debug -15> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -14> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:32:56 osd07 bash[31674]: debug -13> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:32:56 osd07 bash[31674]: debug -12> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:32:56 osd07 bash[31674]: debug -11> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -10> 2021-02-04T08:32:56.101+0000 7f347908af40 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:32:56 osd07 bash[31674]: debug -9> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:32:56 osd07 bash[31674]: debug -8> 2021-02-04T08:32:56.101+0000 7f346b583700 10 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:32:56 osd07 bash[31674]: debug -7> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:32:56 osd07 bash[31674]: debug -6> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:32:56 osd07 bash[31674]: debug -5> 2021-02-04T08:32:56.101+0000 7f347908af40 1 bluefs mount Feb 04 09:32:56 osd07 bash[31674]: debug -4> 2021-02-04T08:32:56.101+0000 7f347908af40 10 bluefs _open_super Feb 04 09:32:56 osd07 bash[31674]: debug -3> 2021-02-04T08:32:56.101+0000 7f347908af40 5 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:32:56 osd07 bash[31674]: debug -2> 2021-02-04T08:32:56.101+0000 7f346ad82700 20 bdev(0x556cabe2a700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:32:56 osd07 bash[31674]: debug -1> 2021-02-04T08:32:56.105+0000 7f347908af40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f347908af40 time 2021-02-04T08:32:56.104984+0000 Feb 04 09:32:56 osd07 bash[31674]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x556c9f6e2372] Feb 04 09:32:56 osd07 bash[31674]: 2: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 4: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 5: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 10: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 11: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 12: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 13: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: debug 0> 2021-02-04T08:32:56.109+0000 7f347908af40 -1 *** Caught signal (Aborted) ** Feb 04 09:32:56 osd07 bash[31674]: in thread 7f347908af40 thread_name:ceph-osd Feb 04 09:32:56 osd07 bash[31674]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:32:56 osd07 bash[31674]: 1: (()+0x12b20) [0x7f3476df2b20] Feb 04 09:32:56 osd07 bash[31674]: 2: (gsignal()+0x10f) [0x7f3475a5a7ff] Feb 04 09:32:56 osd07 bash[31674]: 3: (abort()+0x127) [0x7f3475a44c35] Feb 04 09:32:56 osd07 bash[31674]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x556c9f6e23c3] Feb 04 09:32:56 osd07 bash[31674]: 5: (()+0x50658c) [0x556c9f6e258c] Feb 04 09:32:56 osd07 bash[31674]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x556c9fdbac3d] Feb 04 09:32:56 osd07 bash[31674]: 7: (BlueFS::_open_super()+0x155) [0x556c9fd5a0e5] Feb 04 09:32:56 osd07 bash[31674]: 8: (BlueFS::mount()+0xe9) [0x556c9fd7ef99] Feb 04 09:32:56 osd07 bash[31674]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x556c9fc5c974] Feb 04 09:32:56 osd07 bash[31674]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x556c9fc5de1c] Feb 04 09:32:56 osd07 bash[31674]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x556c9fc70d6c] Feb 04 09:32:56 osd07 bash[31674]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x556c9fcc9007] Feb 04 09:32:56 osd07 bash[31674]: 13: (OSD::init()+0x380) [0x556c9f7f0f20] Feb 04 09:32:56 osd07 bash[31674]: 14: (main()+0x47e4) [0x556c9f744264] Feb 04 09:32:56 osd07 bash[31674]: 15: (__libc_start_main()+0xf3) [0x7f3475a467b3] Feb 04 09:32:56 osd07 bash[31674]: 16: (_start()+0x2e) [0x556c9f77256e] Feb 04 09:32:56 osd07 bash[31674]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:32:56 osd07 bash[31674]: --- logging levels --- Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 none Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 lockdep Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 context Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 crush Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_balancer Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_locker Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_log Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_log_expire Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mds_migrator Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 buffer Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 timer Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 filer Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 striper Feb 04 09:32:56 osd07 bash[31674]: 0/ 1 objecter Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rados Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd_mirror Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd_replay Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 rbd_rwl Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 journaler Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 objectcacher Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 immutable_obj_cache Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 client Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 osd Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 optracker Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 objclass Feb 04 09:32:56 osd07 bash[31674]: 1/ 3 filestore Feb 04 09:32:56 osd07 bash[31674]: 1/ 3 journal Feb 04 09:32:56 osd07 bash[31674]: 0/ 0 ms Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mon Feb 04 09:32:56 osd07 bash[31674]: 0/10 monc Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 paxos Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 tp Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 auth Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 crypto Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 finisher Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 reserver Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 heartbeatmap Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 perfcounter Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 rgw Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 rgw_sync Feb 04 09:32:56 osd07 bash[31674]: 1/10 civetweb Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 javaclient Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 asok Feb 04 09:32:56 osd07 bash[31674]: 1/ 1 throttle Feb 04 09:32:56 osd07 bash[31674]: 0/ 0 refs Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 compressor Feb 04 09:32:56 osd07 bash[31674]: 20/20 bluestore Feb 04 09:32:56 osd07 bash[31674]: 20/20 bluefs Feb 04 09:32:56 osd07 bash[31674]: 20/20 bdev Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 kstore Feb 04 09:32:56 osd07 bash[31674]: 4/ 5 rocksdb Feb 04 09:32:56 osd07 bash[31674]: 4/ 5 leveldb Feb 04 09:32:56 osd07 bash[31674]: 4/ 5 memdb Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 fuse Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mgr Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 mgrc Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 dpdk Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 eventtrace Feb 04 09:32:56 osd07 bash[31674]: 1/ 5 prioritycache Feb 04 09:32:56 osd07 bash[31674]: 0/ 5 test Feb 04 09:32:56 osd07 bash[31674]: -2/-2 (syslog threshold) Feb 04 09:32:56 osd07 bash[31674]: 99/99 (stderr threshold) Feb 04 09:32:56 osd07 bash[31674]: --- pthread ID / name mapping for recent threads --- Feb 04 09:32:56 osd07 bash[31674]: 7f346ad82700 / bstore_discard Feb 04 09:32:56 osd07 bash[31674]: 7f346b583700 / bstore_aio Feb 04 09:32:56 osd07 bash[31674]: 7f346bd84700 / bstore_discard Feb 04 09:32:56 osd07 bash[31674]: 7f346c585700 / bstore_aio Feb 04 09:32:56 osd07 bash[31674]: 7f346f5fc700 / OpHistorySvc Feb 04 09:32:56 osd07 bash[31674]: 7f346fdfd700 / safe_timer Feb 04 09:32:56 osd07 bash[31674]: 7f34705fe700 / signal_handler Feb 04 09:32:56 osd07 bash[31674]: 7f3470dff700 / ceph-osd Feb 04 09:32:56 osd07 bash[31674]: 7f3471600700 / admin_socket Feb 04 09:32:56 osd07 bash[31674]: 7f3471e01700 / service Feb 04 09:32:56 osd07 bash[31674]: 7f3472602700 / msgr-worker-2 Feb 04 09:32:56 osd07 bash[31674]: 7f3472e03700 / msgr-worker-1 Feb 04 09:32:56 osd07 bash[31674]: 7f3473604700 / msgr-worker-0 Feb 04 09:32:56 osd07 bash[31674]: 7f347908af40 / ceph-osd Feb 04 09:32:56 osd07 bash[31674]: max_recent 10000 Feb 04 09:32:56 osd07 bash[31674]: max_new 1000 Feb 04 09:32:56 osd07 bash[31674]: log_file /var/lib/ceph/crash/2021-02-04T08:32:56.112820Z_dcd6beae-2068-4a15-aae2-dcba4a03a090/log Feb 04 09:32:56 osd07 bash[31674]: --- end dump of recent events --- Feb 04 09:32:56 osd07 bash[31674]: reraise_fatal: default handler for signal 6 didn't terminate the process? Feb 04 09:32:56 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Main process exited, code=exited, status=1/FAILURE Feb 04 09:32:56 osd07 bash[31894]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:56 osd07 bash[31894]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-deactivate Feb 04 09:32:56 osd07 bash[31894]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:32:57 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Failed with result 'exit-code'. Feb 04 09:33:07 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Service RestartSec=10s expired, scheduling restart. Feb 04 09:33:07 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Scheduled restart job, restart counter is at 4. Feb 04 09:33:07 osd07 systemd[1]: Stopped Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:33:07 osd07 systemd[1]: Starting Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e... Feb 04 09:33:07 osd07 docker[31994]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:07 osd07 docker[31994]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:33:07 osd07 systemd[1]: Started Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:33:07 osd07 bash[32010]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:07 osd07 bash[32010]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-activate Feb 04 09:33:07 osd07 bash[32010]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:08 osd07 bash[32010]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:33:08 osd07 bash[32010]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 --path /var/lib/ceph/osd/ceph-18 --no-mon-config Feb 04 09:33:08 osd07 bash[32010]: Running command: /usr/bin/ln -snf /dev/ceph-4f944631-bf3e-425e-9e1d-d0f9036d051f/osd-block-579892dd-22bb-4690-835e-a37f5ede1771 /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:08 osd07 bash[32010]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:08 osd07 bash[32010]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7 Feb 04 09:33:08 osd07 bash[32010]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-18 Feb 04 09:33:08 osd07 bash[32010]: --> ceph-volume lvm activate successful for osd ID: 18 Feb 04 09:33:08 osd07 bash[32010]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:08 osd07 bash[32010]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18 Feb 04 09:33:08 osd07 bash[32010]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 pidfile_write: ignore empty --pid-file Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52d8173700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52d7972700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52d7171700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev aio_wait 0x5605c4b48bb0 done Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.273+0000 7f52d7972700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.317+0000 7f52d7171700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.317+0000 7f52d7171700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.317+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.525+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52d8173700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52d8173700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52e0bfff40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.557+0000 7f52e0bfff40 -1 Falling back to public interface Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.569+0000 7f52e0bfff40 0 load: jerasure load: lrc load: isa Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.569+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.569+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52d7171700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.821+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52d7171700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52d7171700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52d40fa700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52d38f9700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52d30f8700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52d28f7700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f52e0bfff40 time 2021-02-04T08:33:09.861329+0000 Feb 04 09:33:09 osd07 bash[32010]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluefs mount Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluefs _open_super Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5605ba0e0372] Feb 04 09:33:09 osd07 bash[32010]: 2: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 4: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 5: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 10: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 11: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 12: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 13: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: *** Caught signal (Aborted) ** Feb 04 09:33:09 osd07 bash[32010]: in thread 7f52e0bfff40 thread_name:ceph-osd Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.861+0000 7f52e0bfff40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f52e0bfff40 time 2021-02-04T08:33:09.861329+0000 Feb 04 09:33:09 osd07 bash[32010]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5605ba0e0372] Feb 04 09:33:09 osd07 bash[32010]: 2: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 4: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 5: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 10: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 11: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 12: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 13: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (()+0x12b20) [0x7f52de967b20] Feb 04 09:33:09 osd07 bash[32010]: 2: (gsignal()+0x10f) [0x7f52dd5cf7ff] Feb 04 09:33:09 osd07 bash[32010]: 3: (abort()+0x127) [0x7f52dd5b9c35] Feb 04 09:33:09 osd07 bash[32010]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5605ba0e03c3] Feb 04 09:33:09 osd07 bash[32010]: 5: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 13: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 14: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 15: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 16: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: debug 2021-02-04T08:33:09.865+0000 7f52e0bfff40 -1 *** Caught signal (Aborted) ** Feb 04 09:33:09 osd07 bash[32010]: in thread 7f52e0bfff40 thread_name:ceph-osd Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (()+0x12b20) [0x7f52de967b20] Feb 04 09:33:09 osd07 bash[32010]: 2: (gsignal()+0x10f) [0x7f52dd5cf7ff] Feb 04 09:33:09 osd07 bash[32010]: 3: (abort()+0x127) [0x7f52dd5b9c35] Feb 04 09:33:09 osd07 bash[32010]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5605ba0e03c3] Feb 04 09:33:09 osd07 bash[32010]: 5: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 13: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 14: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 15: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 16: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:33:09 osd07 bash[32010]: --- begin dump of recent events --- Feb 04 09:33:09 osd07 bash[32010]: debug -378> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command assert hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -377> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command abort hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -376> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command leak_some_memory hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -375> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perfcounters_dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -374> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command 1 hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -373> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -372> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perfcounters_schema hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -371> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf histogram dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -370> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command 2 hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -369> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf schema hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -368> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf histogram schema hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -367> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf reset hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -366> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config show hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -365> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config help hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -364> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config set hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -363> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config unset hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -362> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config get hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -361> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config diff hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -360> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config diff get hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -359> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command injectargs hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -358> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command log flush hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -357> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command log dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -356> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command log reopen hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -355> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command dump_mempools hook 0x5605c5794068 Feb 04 09:33:09 osd07 bash[32010]: debug -354> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: get_monmap_and_config Feb 04 09:33:09 osd07 bash[32010]: debug -353> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: build_initial_monmap Feb 04 09:33:09 osd07 bash[32010]: debug -352> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: monmap: Feb 04 09:33:09 osd07 bash[32010]: epoch 0 Feb 04 09:33:09 osd07 bash[32010]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:33:09 osd07 bash[32010]: last_changed 2021-02-04T08:33:09.020195+0000 Feb 04 09:33:09 osd07 bash[32010]: created 2021-02-04T08:33:09.020195+0000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 0 (unknown) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:33:09 osd07 bash[32010]: debug -351> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -350> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -349> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -348> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -347> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -346> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -345> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -344> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -343> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -342> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -341> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -340> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -339> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -338> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -337> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -336> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -335> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -334> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: init Feb 04 09:33:09 osd07 bash[32010]: debug -333> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -332> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -331> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -330> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -329> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -328> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -327> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -326> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -325> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -324> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -323> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -322> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -321> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -320> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -319> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -318> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -317> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -316> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -315> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: _reopen_session rank -1 Feb 04 09:33:09 osd07 bash[32010]: debug -314> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: _add_conns ranks=[0,2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -313> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): picked mon.noname-a con 0x5605c4bbec00 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:33:09 osd07 bash[32010]: debug -312> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): picked mon.noname-c con 0x5605c4bbf000 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:33:09 osd07 bash[32010]: debug -311> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): picked mon.noname-b con 0x5605c4bbf400 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:33:09 osd07 bash[32010]: debug -310> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): start opening mon connection Feb 04 09:33:09 osd07 bash[32010]: debug -309> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): start opening mon connection Feb 04 09:33:09 osd07 bash[32010]: debug -308> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): start opening mon connection Feb 04 09:33:09 osd07 bash[32010]: debug -307> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): _renew_subs Feb 04 09:33:09 osd07 bash[32010]: debug -306> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:38:09.021452+0000 Feb 04 09:33:09 osd07 bash[32010]: debug -305> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): get_auth_request con 0x5605c4bbec00 auth_method 0 Feb 04 09:33:09 osd07 bash[32010]: debug -304> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -303> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): _init_auth method 2 Feb 04 09:33:09 osd07 bash[32010]: debug -302> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): get_auth_request con 0x5605c4bbf400 auth_method 0 Feb 04 09:33:09 osd07 bash[32010]: debug -301> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -300> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): _init_auth method 2 Feb 04 09:33:09 osd07 bash[32010]: debug -299> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): get_auth_request con 0x5605c4bbf000 auth_method 0 Feb 04 09:33:09 osd07 bash[32010]: debug -298> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -297> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): _init_auth method 2 Feb 04 09:33:09 osd07 bash[32010]: debug -296> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:33:09 osd07 bash[32010]: debug -295> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:33:09 osd07 bash[32010]: debug -294> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:33:09 osd07 bash[32010]: debug -293> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:33:09 osd07 bash[32010]: debug -292> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:33:09 osd07 bash[32010]: debug -291> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:33:09 osd07 bash[32010]: debug -290> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:33:09 osd07 bash[32010]: debug -289> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:33:09 osd07 bash[32010]: debug -288> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:33:09 osd07 bash[32010]: debug -287> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_done global_id 3644957 payload 450 Feb 04 09:33:09 osd07 bash[32010]: debug -286> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient: _finish_hunting 0 Feb 04 09:33:09 osd07 bash[32010]: debug -285> 2021-02-04T08:33:09.021+0000 7f52db179700 1 monclient: found mon.noname-b Feb 04 09:33:09 osd07 bash[32010]: debug -284> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.15:3300/0 Feb 04 09:33:09 osd07 bash[32010]: debug -283> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:33:09 osd07 bash[32010]: debug -282> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: got monmap 9 from mon.noname-b (according to old e9) Feb 04 09:33:09 osd07 bash[32010]: debug -281> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: dump: Feb 04 09:33:09 osd07 bash[32010]: epoch 9 Feb 04 09:33:09 osd07 bash[32010]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:33:09 osd07 bash[32010]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:33:09 osd07 bash[32010]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 15 (octopus) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:33:09 osd07 bash[32010]: debug -280> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: _finish_auth 0 Feb 04 09:33:09 osd07 bash[32010]: debug -279> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:32:39.023274+0000) Feb 04 09:33:09 osd07 bash[32010]: debug -278> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: _send_mon_message to mon.osd05 at v2:10.10.0.15:3300/0 Feb 04 09:33:09 osd07 bash[32010]: debug -277> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 monclient: authenticate success, global_id 3644957 Feb 04 09:33:09 osd07 bash[32010]: debug -276> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:33:09 osd07 bash[32010]: debug -275> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:33:09 osd07 bash[32010]: debug -274> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: got monmap 9 from mon.osd05 (according to old e9) Feb 04 09:33:09 osd07 bash[32010]: debug -273> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: dump: Feb 04 09:33:09 osd07 bash[32010]: epoch 9 Feb 04 09:33:09 osd07 bash[32010]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:33:09 osd07 bash[32010]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:33:09 osd07 bash[32010]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 15 (octopus) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:33:09 osd07 bash[32010]: debug -272> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 monclient: get_monmap_and_config success Feb 04 09:33:09 osd07 bash[32010]: debug -271> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 monclient: shutdown Feb 04 09:33:09 osd07 bash[32010]: debug -270> 2021-02-04T08:33:09.021+0000 7f52d8173700 4 set_mon_vals no callback set Feb 04 09:33:09 osd07 bash[32010]: debug -269> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:33:09 osd07 bash[32010]: debug -268> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:33:09 osd07 bash[32010]: debug -267> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:33:09 osd07 bash[32010]: debug -266> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:33:09 osd07 bash[32010]: debug -265> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:33:09 osd07 bash[32010]: debug -264> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:33:09 osd07 bash[32010]: debug -263> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:33:09 osd07 bash[32010]: debug -262> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 pidfile_write: ignore empty --pid-file Feb 04 09:33:09 osd07 bash[32010]: debug -261> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:33:09 osd07 bash[32010]: debug -260> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:33:09 osd07 bash[32010]: debug -259> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command 0 hook 0x5605c4b18388 Feb 04 09:33:09 osd07 bash[32010]: debug -258> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command version hook 0x5605c4b18388 Feb 04 09:33:09 osd07 bash[32010]: debug -257> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command git_version hook 0x5605c4b18388 Feb 04 09:33:09 osd07 bash[32010]: debug -256> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command help hook 0x5605c4b14370 Feb 04 09:33:09 osd07 bash[32010]: debug -255> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command get_command_descriptions hook 0x5605c4b14070 Feb 04 09:33:09 osd07 bash[32010]: debug -254> 2021-02-04T08:33:09.021+0000 7f52d9175700 5 asok(0x5605c4bc0000) entry start Feb 04 09:33:09 osd07 bash[32010]: debug -253> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 object store type is bluestore Feb 04 09:33:09 osd07 bash[32010]: debug -252> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:33:09 osd07 bash[32010]: debug -251> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -250> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -249> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -248> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -247> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -246> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -245> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -244> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -243> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -242> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -241> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -240> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -239> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:33:09 osd07 bash[32010]: debug -238> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -237> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -236> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -235> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -234> 2021-02-04T08:33:09.025+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -233> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -232> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -231> 2021-02-04T08:33:09.025+0000 7f52d8173700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -230> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -229> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug -228> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluestore bluefs available hook 0x5605c4b140b0 Feb 04 09:33:09 osd07 bash[32010]: debug -227> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluefs stats hook 0x5605c4b140b0 Feb 04 09:33:09 osd07 bash[32010]: debug -226> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -225> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -224> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -223> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -222> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -221> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -220> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -219> 2021-02-04T08:33:09.025+0000 7f52d7972700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -218> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -217> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:33:09 osd07 bash[32010]: debug -216> 2021-02-04T08:33:09.025+0000 7f52d7171700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -215> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) unregister_commands bluefs stats Feb 04 09:33:09 osd07 bash[32010]: debug -214> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) unregister_commands bluestore bluefs available Feb 04 09:33:09 osd07 bash[32010]: debug -213> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev aio_wait 0x5605c4b48bb0 done Feb 04 09:33:09 osd07 bash[32010]: debug -212> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug -211> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug -210> 2021-02-04T08:33:09.273+0000 7f52d7972700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug -209> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug -208> 2021-02-04T08:33:09.317+0000 7f52d7171700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug -207> 2021-02-04T08:33:09.317+0000 7f52d7171700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug -206> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug -205> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug -204> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug -203> 2021-02-04T08:33:09.525+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug -202> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug -201> 2021-02-04T08:33:09.557+0000 7f52d8173700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug -200> 2021-02-04T08:33:09.557+0000 7f52d8173700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug -199> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug -198> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:33:09 osd07 bash[32010]: debug -197> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -196> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -195> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -194> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -193> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -192> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -191> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -190> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -189> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -188> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -187> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -186> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -185> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -184> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -183> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -182> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -181> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -180> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -179> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -178> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -177> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -176> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -175> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -174> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -173> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -172> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -171> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -170> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -169> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -168> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -167> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -166> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -165> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -164> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -163> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -162> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -161> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -160> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -159> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -158> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -157> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -156> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -155> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -154> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -153> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -152> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -151> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -150> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -149> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -148> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -147> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -146> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -145> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -144> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -143> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -142> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -141> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -140> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -139> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -138> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -137> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -136> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -135> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -134> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -133> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -132> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -131> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -130> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -129> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -128> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -127> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -126> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -125> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -124> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -123> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -122> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -121> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -120> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -119> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -118> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -117> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -116> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -115> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -114> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -113> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -112> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -111> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -110> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -109> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -108> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -107> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -106> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -105> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -104> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -103> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -102> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -101> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -100> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -99> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -98> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -97> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -96> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -95> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -94> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -93> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -92> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -91> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -90> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -89> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -88> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -87> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -86> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -85> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -84> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -83> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -82> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -81> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -80> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -79> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -78> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:33:09 osd07 bash[32010]: debug -77> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 -1 Falling back to public interface Feb 04 09:33:09 osd07 bash[32010]: debug -76> 2021-02-04T08:33:09.561+0000 7f52e0bfff40 10 monclient: build_initial_monmap Feb 04 09:33:09 osd07 bash[32010]: debug -75> 2021-02-04T08:33:09.561+0000 7f52e0bfff40 10 monclient: monmap: Feb 04 09:33:09 osd07 bash[32010]: epoch 0 Feb 04 09:33:09 osd07 bash[32010]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:33:09 osd07 bash[32010]: last_changed 0.000000 Feb 04 09:33:09 osd07 bash[32010]: created 0.000000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 0 (unknown) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:33:09 osd07 bash[32010]: debug -74> 2021-02-04T08:33:09.569+0000 7f52e0bfff40 0 load: jerasure load: lrc load: isa Feb 04 09:33:09 osd07 bash[32010]: debug -73> 2021-02-04T08:33:09.569+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -72> 2021-02-04T08:33:09.569+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -71> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:33:09 osd07 bash[32010]: debug -70> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -69> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -68> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -67> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -66> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -65> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -64> 2021-02-04T08:33:09.573+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -63> 2021-02-04T08:33:09.573+0000 7f52d7171700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -62> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -61> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug -60> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug -59> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug -58> 2021-02-04T08:33:09.821+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug -57> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug -56> 2021-02-04T08:33:09.857+0000 7f52d7171700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug -55> 2021-02-04T08:33:09.857+0000 7f52d7171700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug -54> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug -53> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command objecter_requests hook 0x5605c4b14970 Feb 04 09:33:09 osd07 bash[32010]: debug -52> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -51> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -50> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -49> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -48> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -47> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -46> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -45> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:33:09 osd07 bash[32010]: debug -44> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:33:09 osd07 bash[32010]: debug -43> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:33:09 osd07 bash[32010]: debug -42> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:33:09 osd07 bash[32010]: debug -41> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -40> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -39> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -38> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -37> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:33:09 osd07 bash[32010]: debug -36> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -35> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -34> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -33> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -32> 2021-02-04T08:33:09.857+0000 7f52d40fa700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -31> 2021-02-04T08:33:09.857+0000 7f52d38f9700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -30> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -29> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -28> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -27> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug -26> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -25> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -24> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -23> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -22> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:33:09 osd07 bash[32010]: debug -21> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -20> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -19> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:33:09 osd07 bash[32010]: debug -18> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:33:09 osd07 bash[32010]: debug -17> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluestore bluefs available hook 0x5605c4b14980 Feb 04 09:33:09 osd07 bash[32010]: debug -16> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluefs stats hook 0x5605c4b14980 Feb 04 09:33:09 osd07 bash[32010]: debug -15> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -14> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -13> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -12> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -11> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -10> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -9> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -8> 2021-02-04T08:33:09.857+0000 7f52d30f8700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -7> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -6> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:33:09 osd07 bash[32010]: debug -5> 2021-02-04T08:33:09.857+0000 7f52d28f7700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -4> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluefs mount Feb 04 09:33:09 osd07 bash[32010]: debug -3> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluefs _open_super Feb 04 09:33:09 osd07 bash[32010]: debug -2> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:33:09 osd07 bash[32010]: debug -1> 2021-02-04T08:33:09.861+0000 7f52e0bfff40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f52e0bfff40 time 2021-02-04T08:33:09.861329+0000 Feb 04 09:33:09 osd07 bash[32010]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5605ba0e0372] Feb 04 09:33:09 osd07 bash[32010]: 2: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 4: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 5: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 10: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 11: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 12: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 13: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: debug 0> 2021-02-04T08:33:09.865+0000 7f52e0bfff40 -1 *** Caught signal (Aborted) ** Feb 04 09:33:09 osd07 bash[32010]: in thread 7f52e0bfff40 thread_name:ceph-osd Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (()+0x12b20) [0x7f52de967b20] Feb 04 09:33:09 osd07 bash[32010]: 2: (gsignal()+0x10f) [0x7f52dd5cf7ff] Feb 04 09:33:09 osd07 bash[32010]: 3: (abort()+0x127) [0x7f52dd5b9c35] Feb 04 09:33:09 osd07 bash[32010]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5605ba0e03c3] Feb 04 09:33:09 osd07 bash[32010]: 5: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 13: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 14: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 15: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 16: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:33:09 osd07 bash[32010]: --- logging levels --- Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 none Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 lockdep Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 context Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 crush Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_balancer Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_locker Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_log Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_log_expire Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_migrator Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 buffer Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 timer Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 filer Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 striper Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 objecter Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rados Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd_mirror Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd_replay Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd_rwl Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 journaler Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 objectcacher Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 immutable_obj_cache Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 client Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 osd Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 optracker Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 objclass Feb 04 09:33:09 osd07 bash[32010]: 1/ 3 filestore Feb 04 09:33:09 osd07 bash[32010]: 1/ 3 journal Feb 04 09:33:09 osd07 bash[32010]: 0/ 0 ms Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mon Feb 04 09:33:09 osd07 bash[32010]: 0/10 monc Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 paxos Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 tp Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 auth Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 crypto Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 finisher Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 reserver Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 heartbeatmap Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 perfcounter Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 rgw Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 rgw_sync Feb 04 09:33:09 osd07 bash[32010]: 1/10 civetweb Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 javaclient Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 asok Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 throttle Feb 04 09:33:09 osd07 bash[32010]: 0/ 0 refs Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 compressor Feb 04 09:33:09 osd07 bash[32010]: 20/20 bluestore Feb 04 09:33:09 osd07 bash[32010]: 20/20 bluefs Feb 04 09:33:09 osd07 bash[32010]: 20/20 bdev Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 kstore Feb 04 09:33:09 osd07 bash[32010]: 4/ 5 rocksdb Feb 04 09:33:09 osd07 bash[32010]: 4/ 5 leveldb Feb 04 09:33:09 osd07 bash[32010]: 4/ 5 memdb Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 fuse Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mgr Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mgrc Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 dpdk Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 eventtrace Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 prioritycache Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 test Feb 04 09:33:09 osd07 bash[32010]: -2/-2 (syslog threshold) Feb 04 09:33:09 osd07 bash[32010]: 99/99 (stderr threshold) Feb 04 09:33:09 osd07 bash[32010]: --- pthread ID / name mapping for recent threads --- Feb 04 09:33:09 osd07 bash[32010]: 7f52d28f7700 / bstore_discard Feb 04 09:33:09 osd07 bash[32010]: 7f52d30f8700 / bstore_aio Feb 04 09:33:09 osd07 bash[32010]: 7f52d38f9700 / bstore_discard Feb 04 09:33:09 osd07 bash[32010]: 7f52d40fa700 / bstore_aio Feb 04 09:33:09 osd07 bash[32010]: 7f52d7171700 / OpHistorySvc Feb 04 09:33:09 osd07 bash[32010]: 7f52d7972700 / safe_timer Feb 04 09:33:09 osd07 bash[32010]: 7f52d8173700 / signal_handler Feb 04 09:33:09 osd07 bash[32010]: 7f52d8974700 / ceph-osd Feb 04 09:33:09 osd07 bash[32010]: 7f52d9175700 / admin_socket Feb 04 09:33:09 osd07 bash[32010]: 7f52d9976700 / service Feb 04 09:33:09 osd07 bash[32010]: 7f52da177700 / msgr-worker-2 Feb 04 09:33:09 osd07 bash[32010]: 7f52da978700 / msgr-worker-1 Feb 04 09:33:09 osd07 bash[32010]: 7f52db179700 / msgr-worker-0 Feb 04 09:33:09 osd07 bash[32010]: 7f52e0bfff40 / ceph-osd Feb 04 09:33:09 osd07 bash[32010]: max_recent 10000 Feb 04 09:33:09 osd07 bash[32010]: max_new 1000 Feb 04 09:33:09 osd07 bash[32010]: log_file Feb 04 09:33:09 osd07 bash[32010]: --- end dump of recent events --- Feb 04 09:33:09 osd07 bash[32010]: --- begin dump of recent events --- Feb 04 09:33:09 osd07 bash[32010]: debug -378> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command assert hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -377> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command abort hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -376> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command leak_some_memory hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -375> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perfcounters_dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -374> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command 1 hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -373> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -372> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perfcounters_schema hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -371> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf histogram dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -370> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command 2 hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -369> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf schema hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -368> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf histogram schema hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -367> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command perf reset hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -366> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config show hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -365> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config help hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -364> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config set hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -363> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config unset hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -362> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config get hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -361> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config diff hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -360> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command config diff get hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -359> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command injectargs hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -358> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command log flush hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -357> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command log dump hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -356> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command log reopen hook 0x5605c4b145b0 Feb 04 09:33:09 osd07 bash[32010]: debug -355> 2021-02-04T08:33:09.013+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command dump_mempools hook 0x5605c5794068 Feb 04 09:33:09 osd07 bash[32010]: debug -354> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: get_monmap_and_config Feb 04 09:33:09 osd07 bash[32010]: debug -353> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: build_initial_monmap Feb 04 09:33:09 osd07 bash[32010]: debug -352> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: monmap: Feb 04 09:33:09 osd07 bash[32010]: epoch 0 Feb 04 09:33:09 osd07 bash[32010]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:33:09 osd07 bash[32010]: last_changed 2021-02-04T08:33:09.020195+0000 Feb 04 09:33:09 osd07 bash[32010]: created 2021-02-04T08:33:09.020195+0000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 0 (unknown) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:33:09 osd07 bash[32010]: debug -351> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -350> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -349> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -348> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -347> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -346> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -345> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -344> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -343> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -342> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -341> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -340> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -339> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -338> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -337> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -336> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4baa940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -335> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -334> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: init Feb 04 09:33:09 osd07 bash[32010]: debug -333> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -332> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -331> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -330> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -329> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -328> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -327> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -326> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -325> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -324> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -323> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -322> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -321> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -320> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -319> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -318> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 5 AuthRegistry(0x7ffe80839a60) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -317> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -316> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -315> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: _reopen_session rank -1 Feb 04 09:33:09 osd07 bash[32010]: debug -314> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient: _add_conns ranks=[0,2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -313> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): picked mon.noname-a con 0x5605c4bbec00 addr [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] Feb 04 09:33:09 osd07 bash[32010]: debug -312> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): picked mon.noname-c con 0x5605c4bbf000 addr [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] Feb 04 09:33:09 osd07 bash[32010]: debug -311> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): picked mon.noname-b con 0x5605c4bbf400 addr [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] Feb 04 09:33:09 osd07 bash[32010]: debug -310> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): start opening mon connection Feb 04 09:33:09 osd07 bash[32010]: debug -309> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): start opening mon connection Feb 04 09:33:09 osd07 bash[32010]: debug -308> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): start opening mon connection Feb 04 09:33:09 osd07 bash[32010]: debug -307> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): _renew_subs Feb 04 09:33:09 osd07 bash[32010]: debug -306> 2021-02-04T08:33:09.017+0000 7f52e0bfff40 10 monclient(hunting): authenticate will time out at 2021-02-04T08:38:09.021452+0000 Feb 04 09:33:09 osd07 bash[32010]: debug -305> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): get_auth_request con 0x5605c4bbec00 auth_method 0 Feb 04 09:33:09 osd07 bash[32010]: debug -304> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -303> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): _init_auth method 2 Feb 04 09:33:09 osd07 bash[32010]: debug -302> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): get_auth_request con 0x5605c4bbf400 auth_method 0 Feb 04 09:33:09 osd07 bash[32010]: debug -301> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -300> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): _init_auth method 2 Feb 04 09:33:09 osd07 bash[32010]: debug -299> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): get_auth_request con 0x5605c4bbf000 auth_method 0 Feb 04 09:33:09 osd07 bash[32010]: debug -298> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] Feb 04 09:33:09 osd07 bash[32010]: debug -297> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): _init_auth method 2 Feb 04 09:33:09 osd07 bash[32010]: debug -296> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:33:09 osd07 bash[32010]: debug -295> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:33:09 osd07 bash[32010]: debug -294> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:33:09 osd07 bash[32010]: debug -293> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:33:09 osd07 bash[32010]: debug -292> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:33:09 osd07 bash[32010]: debug -291> 2021-02-04T08:33:09.021+0000 7f52da978700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:33:09 osd07 bash[32010]: debug -290> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): handle_auth_reply_more payload 9 Feb 04 09:33:09 osd07 bash[32010]: debug -289> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): handle_auth_reply_more payload_len 9 Feb 04 09:33:09 osd07 bash[32010]: debug -288> 2021-02-04T08:33:09.021+0000 7f52da177700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes Feb 04 09:33:09 osd07 bash[32010]: debug -287> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient(hunting): handle_auth_done global_id 3644957 payload 450 Feb 04 09:33:09 osd07 bash[32010]: debug -286> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient: _finish_hunting 0 Feb 04 09:33:09 osd07 bash[32010]: debug -285> 2021-02-04T08:33:09.021+0000 7f52db179700 1 monclient: found mon.noname-b Feb 04 09:33:09 osd07 bash[32010]: debug -284> 2021-02-04T08:33:09.021+0000 7f52db179700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.15:3300/0 Feb 04 09:33:09 osd07 bash[32010]: debug -283> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:33:09 osd07 bash[32010]: debug -282> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: got monmap 9 from mon.noname-b (according to old e9) Feb 04 09:33:09 osd07 bash[32010]: debug -281> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: dump: Feb 04 09:33:09 osd07 bash[32010]: epoch 9 Feb 04 09:33:09 osd07 bash[32010]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:33:09 osd07 bash[32010]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:33:09 osd07 bash[32010]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 15 (octopus) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:33:09 osd07 bash[32010]: debug -280> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: _finish_auth 0 Feb 04 09:33:09 osd07 bash[32010]: debug -279> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-02-04T08:32:39.023274+0000) Feb 04 09:33:09 osd07 bash[32010]: debug -278> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: _send_mon_message to mon.osd05 at v2:10.10.0.15:3300/0 Feb 04 09:33:09 osd07 bash[32010]: debug -277> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 monclient: authenticate success, global_id 3644957 Feb 04 09:33:09 osd07 bash[32010]: debug -276> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: handle_config config(5 keys) v1 Feb 04 09:33:09 osd07 bash[32010]: debug -275> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: handle_monmap mon_map magic: 0 v1 Feb 04 09:33:09 osd07 bash[32010]: debug -274> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: got monmap 9 from mon.osd05 (according to old e9) Feb 04 09:33:09 osd07 bash[32010]: debug -273> 2021-02-04T08:33:09.021+0000 7f52d9976700 10 monclient: dump: Feb 04 09:33:09 osd07 bash[32010]: epoch 9 Feb 04 09:33:09 osd07 bash[32010]: fsid 8fde54d0-45e9-11eb-86ab-a23d47ea900e Feb 04 09:33:09 osd07 bash[32010]: last_changed 2020-12-29T20:10:09.617148+0000 Feb 04 09:33:09 osd07 bash[32010]: created 2020-12-24T13:12:03.193261+0000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 15 (octopus) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.osd04 Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.osd06 Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.osd05 Feb 04 09:33:09 osd07 bash[32010]: debug -272> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 monclient: get_monmap_and_config success Feb 04 09:33:09 osd07 bash[32010]: debug -271> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 monclient: shutdown Feb 04 09:33:09 osd07 bash[32010]: debug -270> 2021-02-04T08:33:09.021+0000 7f52d8173700 4 set_mon_vals no callback set Feb 04 09:33:09 osd07 bash[32010]: debug -269> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals bdev_block_size = 32768 Feb 04 09:33:09 osd07 bash[32010]: debug -268> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals container_image = docker.io/ceph/ceph:v15 Feb 04 09:33:09 osd07 bash[32010]: debug -267> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals mon_allow_pool_delete = true Feb 04 09:33:09 osd07 bash[32010]: debug -266> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals osd_pool_default_size = 2 Feb 04 09:33:09 osd07 bash[32010]: debug -265> 2021-02-04T08:33:09.021+0000 7f52d8173700 10 set_mon_vals rbd_config_pool_override_update_timestamp = 1608816456 Feb 04 09:33:09 osd07 bash[32010]: debug -264> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 set uid:gid to 167:167 (ceph:ceph) Feb 04 09:33:09 osd07 bash[32010]: debug -263> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-osd, pid 1 Feb 04 09:33:09 osd07 bash[32010]: debug -262> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 0 pidfile_write: ignore empty --pid-file Feb 04 09:33:09 osd07 bash[32010]: debug -261> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) init /var/run/ceph/ceph-osd.18.asok Feb 04 09:33:09 osd07 bash[32010]: debug -260> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) bind_and_listen /var/run/ceph/ceph-osd.18.asok Feb 04 09:33:09 osd07 bash[32010]: debug -259> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command 0 hook 0x5605c4b18388 Feb 04 09:33:09 osd07 bash[32010]: debug -258> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command version hook 0x5605c4b18388 Feb 04 09:33:09 osd07 bash[32010]: debug -257> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command git_version hook 0x5605c4b18388 Feb 04 09:33:09 osd07 bash[32010]: debug -256> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command help hook 0x5605c4b14370 Feb 04 09:33:09 osd07 bash[32010]: debug -255> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command get_command_descriptions hook 0x5605c4b14070 Feb 04 09:33:09 osd07 bash[32010]: debug -254> 2021-02-04T08:33:09.021+0000 7f52d9175700 5 asok(0x5605c4bc0000) entry start Feb 04 09:33:09 osd07 bash[32010]: debug -253> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 5 object store type is bluestore Feb 04 09:33:09 osd07 bash[32010]: debug -252> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 1 Feb 04 09:33:09 osd07 bash[32010]: debug -251> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -250> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -249> 2021-02-04T08:33:09.021+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -248> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -247> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -246> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -245> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -244> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -243> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -242> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -241> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -240> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -239> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 19 Feb 04 09:33:09 osd07 bash[32010]: debug -238> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -237> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -236> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -235> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -234> 2021-02-04T08:33:09.025+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -233> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -232> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -231> 2021-02-04T08:33:09.025+0000 7f52d8173700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -230> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -229> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug -228> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluestore bluefs available hook 0x5605c4b140b0 Feb 04 09:33:09 osd07 bash[32010]: debug -227> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluefs stats hook 0x5605c4b140b0 Feb 04 09:33:09 osd07 bash[32010]: debug -226> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -225> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -224> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -223> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -222> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -221> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -220> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -219> 2021-02-04T08:33:09.025+0000 7f52d7972700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -218> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -217> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:33:09 osd07 bash[32010]: debug -216> 2021-02-04T08:33:09.025+0000 7f52d7171700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -215> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) unregister_commands bluefs stats Feb 04 09:33:09 osd07 bash[32010]: debug -214> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) unregister_commands bluestore bluefs available Feb 04 09:33:09 osd07 bash[32010]: debug -213> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 20 bdev aio_wait 0x5605c4b48bb0 done Feb 04 09:33:09 osd07 bash[32010]: debug -212> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug -211> 2021-02-04T08:33:09.025+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug -210> 2021-02-04T08:33:09.273+0000 7f52d7972700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug -209> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug -208> 2021-02-04T08:33:09.317+0000 7f52d7171700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug -207> 2021-02-04T08:33:09.317+0000 7f52d7171700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug -206> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug -205> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug -204> 2021-02-04T08:33:09.317+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug -203> 2021-02-04T08:33:09.525+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug -202> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug -201> 2021-02-04T08:33:09.557+0000 7f52d8173700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug -200> 2021-02-04T08:33:09.557+0000 7f52d8173700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug -199> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug -198> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) get_numa_node bdev sda can't detect numa_node Feb 04 09:33:09 osd07 bash[32010]: debug -197> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -196> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -195> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -194> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -193> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -192> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -191> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -190> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -189> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -188> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -187> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -186> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -185> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -184> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -183> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -182> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bab940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -181> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -180> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -179> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -178> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -177> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -176> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -175> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -174> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -173> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -172> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -171> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -170> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -169> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -168> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -167> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -166> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -165> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bac940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -164> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -163> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -162> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -161> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -160> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -159> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -158> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -157> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -156> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -155> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -154> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -153> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -152> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -151> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -150> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -149> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -148> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c4bad940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -147> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -146> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -145> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -144> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -143> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -142> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -141> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -140> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -139> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -138> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -137> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -136> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -135> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -134> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -133> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -132> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -131> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5870940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -130> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -129> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -128> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -127> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -126> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -125> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -124> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -123> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -122> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -121> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -120> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -119> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -118> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -117> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -116> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -115> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -114> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5871940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -113> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -112> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -111> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -110> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -109> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -108> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -107> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -106> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -105> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -104> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -103> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -102> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -101> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -100> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -99> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -98> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -97> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5872940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -96> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -95> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -94> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -93> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: cephx Feb 04 09:33:09 osd07 bash[32010]: debug -92> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding auth protocol: none Feb 04 09:33:09 osd07 bash[32010]: debug -91> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -90> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -89> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -88> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -87> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -86> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -85> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -84> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -83> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -82> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -81> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: crc Feb 04 09:33:09 osd07 bash[32010]: debug -80> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 5 AuthRegistry(0x5605c5873940) adding con mode: secure Feb 04 09:33:09 osd07 bash[32010]: debug -79> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-18/keyring Feb 04 09:33:09 osd07 bash[32010]: debug -78> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 0 starting osd.18 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal Feb 04 09:33:09 osd07 bash[32010]: debug -77> 2021-02-04T08:33:09.557+0000 7f52e0bfff40 -1 Falling back to public interface Feb 04 09:33:09 osd07 bash[32010]: debug -76> 2021-02-04T08:33:09.561+0000 7f52e0bfff40 10 monclient: build_initial_monmap Feb 04 09:33:09 osd07 bash[32010]: debug -75> 2021-02-04T08:33:09.561+0000 7f52e0bfff40 10 monclient: monmap: Feb 04 09:33:09 osd07 bash[32010]: epoch 0 Feb 04 09:33:09 osd07 bash[32010]: fsid 00000000-0000-0000-0000-000000000000 Feb 04 09:33:09 osd07 bash[32010]: last_changed 0.000000 Feb 04 09:33:09 osd07 bash[32010]: created 0.000000 Feb 04 09:33:09 osd07 bash[32010]: min_mon_release 0 (unknown) Feb 04 09:33:09 osd07 bash[32010]: 0: [v2:10.10.0.14:3300/0,v1:10.10.0.14:6789/0] mon.noname-a Feb 04 09:33:09 osd07 bash[32010]: 1: [v2:10.10.0.15:3300/0,v1:10.10.0.15:6789/0] mon.noname-b Feb 04 09:33:09 osd07 bash[32010]: 2: [v2:10.10.0.16:3300/0,v1:10.10.0.16:6789/0] mon.noname-c Feb 04 09:33:09 osd07 bash[32010]: debug -74> 2021-02-04T08:33:09.569+0000 7f52e0bfff40 0 load: jerasure load: lrc load: isa Feb 04 09:33:09 osd07 bash[32010]: debug -73> 2021-02-04T08:33:09.569+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -72> 2021-02-04T08:33:09.569+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -71> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:33:09 osd07 bash[32010]: debug -70> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -69> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -68> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -67> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -66> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -65> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -64> 2021-02-04T08:33:09.573+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -63> 2021-02-04T08:33:09.573+0000 7f52d7171700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -62> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -61> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug -60> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) close Feb 04 09:33:09 osd07 bash[32010]: debug -59> 2021-02-04T08:33:09.573+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_stop Feb 04 09:33:09 osd07 bash[32010]: debug -58> 2021-02-04T08:33:09.821+0000 7f52d8974700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread end Feb 04 09:33:09 osd07 bash[32010]: debug -57> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop Feb 04 09:33:09 osd07 bash[32010]: debug -56> 2021-02-04T08:33:09.857+0000 7f52d7171700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread wake Feb 04 09:33:09 osd07 bash[32010]: debug -55> 2021-02-04T08:33:09.857+0000 7f52d7171700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread finish Feb 04 09:33:09 osd07 bash[32010]: debug -54> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_stop stopped Feb 04 09:33:09 osd07 bash[32010]: debug -53> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command objecter_requests hook 0x5605c4b14970 Feb 04 09:33:09 osd07 bash[32010]: debug -52> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -51> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -50> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -49> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -48> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 0 osd.18:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Feb 04 09:33:09 osd07 bash[32010]: debug -47> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -46> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -45> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 2 osd.18 0 init /var/lib/ceph/osd/ceph-18 (looks like hdd) Feb 04 09:33:09 osd07 bash[32010]: debug -44> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 2 osd.18 0 journal /var/lib/ceph/osd/ceph-18/journal Feb 04 09:33:09 osd07 bash[32010]: debug -43> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) set_cache_shards 32 Feb 04 09:33:09 osd07 bash[32010]: debug -42> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _mount path /var/lib/ceph/osd/ceph-18 Feb 04 09:33:09 osd07 bash[32010]: debug -41> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -40> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -39> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -38> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -37> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _lock 31 Feb 04 09:33:09 osd07 bash[32010]: debug -36> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -35> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -34> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -33> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -32> 2021-02-04T08:33:09.857+0000 7f52d40fa700 10 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -31> 2021-02-04T08:33:09.857+0000 7f52d38f9700 20 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -30> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c000 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -29> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -28> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -27> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluestore(/var/lib/ceph/osd/ceph-18) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 Feb 04 09:33:09 osd07 bash[32010]: debug -26> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -25> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -24> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -23> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -22> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db kv_backend = rocksdb Feb 04 09:33:09 osd07 bash[32010]: debug -21> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label Feb 04 09:33:09 osd07 bash[32010]: debug -20> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18/block) _read_bdev_label got bdev(osd_uuid 579892dd-22bb-4690-835e-a37f5ede1771, size 0xae9ffc00000, btime 2020-12-28T11:20:05.530474+0000, desc main, 14 meta) Feb 04 09:33:09 osd07 bash[32010]: debug -19> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db do_bluefs = 1 Feb 04 09:33:09 osd07 bash[32010]: debug -18> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluestore(/var/lib/ceph/osd/ceph-18) _open_db initializing bluefs Feb 04 09:33:09 osd07 bash[32010]: debug -17> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluestore bluefs available hook 0x5605c4b14980 Feb 04 09:33:09 osd07 bash[32010]: debug -16> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 asok(0x5605c4bc0000) register_command bluefs stats hook 0x5605c4b14980 Feb 04 09:33:09 osd07 bash[32010]: debug -15> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -14> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev create path /var/lib/ceph/osd/ceph-18/block type kernel Feb 04 09:33:09 osd07 bash[32010]: debug -13> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open path /var/lib/ceph/osd/ceph-18/block Feb 04 09:33:09 osd07 bash[32010]: debug -12> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open backing device/file reports st_blksize 4096, using bdev_block_size 32768 anyway Feb 04 09:33:09 osd07 bash[32010]: debug -11> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open devname dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -10> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _detect_vdo no VDO volume maps to dm-7 Feb 04 09:33:09 osd07 bash[32010]: debug -9> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_start Feb 04 09:33:09 osd07 bash[32010]: debug -8> 2021-02-04T08:33:09.857+0000 7f52d30f8700 10 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _aio_thread start Feb 04 09:33:09 osd07 bash[32010]: debug -7> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) open size 12000134430720 (0xae9ffc00000, 11 TiB) block_size 32768 (32 KiB) rotational discard not supported Feb 04 09:33:09 osd07 bash[32010]: debug -6> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-18/block size 11 TiB Feb 04 09:33:09 osd07 bash[32010]: debug -5> 2021-02-04T08:33:09.857+0000 7f52d28f7700 20 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) _discard_thread sleep Feb 04 09:33:09 osd07 bash[32010]: debug -4> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 1 bluefs mount Feb 04 09:33:09 osd07 bash[32010]: debug -3> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 10 bluefs _open_super Feb 04 09:33:09 osd07 bash[32010]: debug -2> 2021-02-04T08:33:09.857+0000 7f52e0bfff40 5 bdev(0x5605c585c700 /var/lib/ceph/osd/ceph-18/block) read 0x1000~1000 (direct) Feb 04 09:33:09 osd07 bash[32010]: debug -1> 2021-02-04T08:33:09.861+0000 7f52e0bfff40 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7f52e0bfff40 time 2021-02-04T08:33:09.861329+0000 Feb 04 09:33:09 osd07 bash[32010]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.8/rpm/el8/BUILD/ceph-15.2.8/src/os/bluestore/KernelDevice.cc: 999: FAILED ceph_assert(is_valid_io(off, len)) Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x5605ba0e0372] Feb 04 09:33:09 osd07 bash[32010]: 2: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 4: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 5: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 6: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 10: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 11: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 12: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 13: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: debug 0> 2021-02-04T08:33:09.865+0000 7f52e0bfff40 -1 *** Caught signal (Aborted) ** Feb 04 09:33:09 osd07 bash[32010]: in thread 7f52e0bfff40 thread_name:ceph-osd Feb 04 09:33:09 osd07 bash[32010]: ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable) Feb 04 09:33:09 osd07 bash[32010]: 1: (()+0x12b20) [0x7f52de967b20] Feb 04 09:33:09 osd07 bash[32010]: 2: (gsignal()+0x10f) [0x7f52dd5cf7ff] Feb 04 09:33:09 osd07 bash[32010]: 3: (abort()+0x127) [0x7f52dd5b9c35] Feb 04 09:33:09 osd07 bash[32010]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x5605ba0e03c3] Feb 04 09:33:09 osd07 bash[32010]: 5: (()+0x50658c) [0x5605ba0e058c] Feb 04 09:33:09 osd07 bash[32010]: 6: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, IOContext*, bool)+0x8d) [0x5605ba7b8c3d] Feb 04 09:33:09 osd07 bash[32010]: 7: (BlueFS::_open_super()+0x155) [0x5605ba7580e5] Feb 04 09:33:09 osd07 bash[32010]: 8: (BlueFS::mount()+0xe9) [0x5605ba77cf99] Feb 04 09:33:09 osd07 bash[32010]: 9: (BlueStore::_open_bluefs(bool)+0x94) [0x5605ba65a974] Feb 04 09:33:09 osd07 bash[32010]: 10: (BlueStore::_open_db(bool, bool, bool)+0x81c) [0x5605ba65be1c] Feb 04 09:33:09 osd07 bash[32010]: 11: (BlueStore::_open_db_and_around(bool)+0x4c) [0x5605ba66ed6c] Feb 04 09:33:09 osd07 bash[32010]: 12: (BlueStore::_mount(bool, bool)+0x847) [0x5605ba6c7007] Feb 04 09:33:09 osd07 bash[32010]: 13: (OSD::init()+0x380) [0x5605ba1eef20] Feb 04 09:33:09 osd07 bash[32010]: 14: (main()+0x47e4) [0x5605ba142264] Feb 04 09:33:09 osd07 bash[32010]: 15: (__libc_start_main()+0xf3) [0x7f52dd5bb7b3] Feb 04 09:33:09 osd07 bash[32010]: 16: (_start()+0x2e) [0x5605ba17056e] Feb 04 09:33:09 osd07 bash[32010]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 04 09:33:09 osd07 bash[32010]: --- logging levels --- Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 none Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 lockdep Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 context Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 crush Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_balancer Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_locker Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_log Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_log_expire Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mds_migrator Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 buffer Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 timer Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 filer Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 striper Feb 04 09:33:09 osd07 bash[32010]: 0/ 1 objecter Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rados Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd_mirror Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd_replay Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 rbd_rwl Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 journaler Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 objectcacher Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 immutable_obj_cache Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 client Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 osd Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 optracker Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 objclass Feb 04 09:33:09 osd07 bash[32010]: 1/ 3 filestore Feb 04 09:33:09 osd07 bash[32010]: 1/ 3 journal Feb 04 09:33:09 osd07 bash[32010]: 0/ 0 ms Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mon Feb 04 09:33:09 osd07 bash[32010]: 0/10 monc Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 paxos Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 tp Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 auth Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 crypto Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 finisher Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 reserver Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 heartbeatmap Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 perfcounter Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 rgw Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 rgw_sync Feb 04 09:33:09 osd07 bash[32010]: 1/10 civetweb Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 javaclient Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 asok Feb 04 09:33:09 osd07 bash[32010]: 1/ 1 throttle Feb 04 09:33:09 osd07 bash[32010]: 0/ 0 refs Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 compressor Feb 04 09:33:09 osd07 bash[32010]: 20/20 bluestore Feb 04 09:33:09 osd07 bash[32010]: 20/20 bluefs Feb 04 09:33:09 osd07 bash[32010]: 20/20 bdev Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 kstore Feb 04 09:33:09 osd07 bash[32010]: 4/ 5 rocksdb Feb 04 09:33:09 osd07 bash[32010]: 4/ 5 leveldb Feb 04 09:33:09 osd07 bash[32010]: 4/ 5 memdb Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 fuse Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mgr Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 mgrc Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 dpdk Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 eventtrace Feb 04 09:33:09 osd07 bash[32010]: 1/ 5 prioritycache Feb 04 09:33:09 osd07 bash[32010]: 0/ 5 test Feb 04 09:33:09 osd07 bash[32010]: -2/-2 (syslog threshold) Feb 04 09:33:09 osd07 bash[32010]: 99/99 (stderr threshold) Feb 04 09:33:09 osd07 bash[32010]: --- pthread ID / name mapping for recent threads --- Feb 04 09:33:09 osd07 bash[32010]: 7f52d28f7700 / bstore_discard Feb 04 09:33:09 osd07 bash[32010]: 7f52d30f8700 / bstore_aio Feb 04 09:33:09 osd07 bash[32010]: 7f52d38f9700 / bstore_discard Feb 04 09:33:09 osd07 bash[32010]: 7f52d40fa700 / bstore_aio Feb 04 09:33:09 osd07 bash[32010]: 7f52d7171700 / OpHistorySvc Feb 04 09:33:09 osd07 bash[32010]: 7f52d7972700 / safe_timer Feb 04 09:33:09 osd07 bash[32010]: 7f52d8173700 / signal_handler Feb 04 09:33:09 osd07 bash[32010]: 7f52d8974700 / ceph-osd Feb 04 09:33:09 osd07 bash[32010]: 7f52d9175700 / admin_socket Feb 04 09:33:09 osd07 bash[32010]: 7f52d9976700 / service Feb 04 09:33:09 osd07 bash[32010]: 7f52da177700 / msgr-worker-2 Feb 04 09:33:09 osd07 bash[32010]: 7f52da978700 / msgr-worker-1 Feb 04 09:33:09 osd07 bash[32010]: 7f52db179700 / msgr-worker-0 Feb 04 09:33:09 osd07 bash[32010]: 7f52e0bfff40 / ceph-osd Feb 04 09:33:09 osd07 bash[32010]: max_recent 10000 Feb 04 09:33:09 osd07 bash[32010]: max_new 1000 Feb 04 09:33:09 osd07 bash[32010]: log_file /var/lib/ceph/crash/2021-02-04T08:33:09.869172Z_cb8aca0a-4b51-44c4-8a69-c1e7baeb1769/log Feb 04 09:33:09 osd07 bash[32010]: --- end dump of recent events --- Feb 04 09:33:09 osd07 bash[32010]: reraise_fatal: default handler for signal 6 didn't terminate the process? Feb 04 09:33:10 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Main process exited, code=exited, status=1/FAILURE Feb 04 09:33:10 osd07 bash[32226]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:10 osd07 bash[32226]: Error: No such container: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e-osd.18-deactivate Feb 04 09:33:10 osd07 bash[32226]: WARNING: Error loading config file: .dockercfg: $HOME is not defined Feb 04 09:33:10 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Failed with result 'exit-code'. Feb 04 09:33:21 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Service RestartSec=10s expired, scheduling restart. Feb 04 09:33:21 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Scheduled restart job, restart counter is at 5. Feb 04 09:33:21 osd07 systemd[1]: Stopped Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e. Feb 04 09:33:21 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Start request repeated too quickly. Feb 04 09:33:21 osd07 systemd[1]: ceph-8fde54d0-45e9-11eb-86ab-a23d47ea900e@osd.18.service: Failed with result 'exit-code'. Feb 04 09:33:21 osd07 systemd[1]: Failed to start Ceph osd.18 for 8fde54d0-45e9-11eb-86ab-a23d47ea900e.