2019-11-07 16:24:10.011 7f873c700dc0 0 set uid:gid to 167:167 (ceph:ceph) 2019-11-07 16:24:10.011 7f873c700dc0 0 ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable), process ceph-osd, pid 2350481 2019-11-07 16:24:10.011 7f873c700dc0 0 pidfile_write: ignore empty --pid-file 2019-11-07 16:24:10.048 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:10.048 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:10.048 7f873c700dc0 0 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:10.049 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:10.049 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-11-07 16:24:10.049 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel 2019-11-07 16:24:10.049 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db 2019-11-07 16:24:10.049 7f873c700dc0 0 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:10.050 7f873c700dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:10.050 7f873c700dc0 0 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:10.050 7f873c700dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB 2019-11-07 16:24:10.051 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel 2019-11-07 16:24:10.051 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal 2019-11-07 16:24:10.051 7f873c700dc0 0 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument 2019-11-07 16:24:10.051 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:10.051 7f873c700dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB 2019-11-07 16:24:10.054 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) close 2019-11-07 16:24:10.308 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) close 2019-11-07 16:24:10.555 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) close 2019-11-07 16:24:10.809 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) close 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553338) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553338) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26552a38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26552a38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding auth protocol: cephx 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553c38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553c38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.063 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554538) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554538) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554e38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554e38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26555738) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26555738) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding auth protocol: cephx 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_method_list cephx -> [2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee2724e138) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee2724e138) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:11.064 7f873c700dc0 0 starting osd.20 osd_data /var/lib/ceph/osd/ceph-20 /var/lib/ceph/osd/ceph-20/journal 2019-11-07 16:24:11.064 7f873c700dc0 -1 Falling back to public interface 2019-11-07 16:24:11.105 7f873c700dc0 0 load: jerasure load: lrc load: isa 2019-11-07 16:24:11.105 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:11.105 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:11.106 7f873c700dc0 0 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:11.106 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:11.106 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-11-07 16:24:11.106 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) close 2019-11-07 16:24:11.371 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _mount path /var/lib/ceph/osd/ceph-20 2019-11-07 16:24:11.371 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:11.371 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:11.371 7f873c700dc0 0 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:11.372 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:11.372 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-11-07 16:24:11.372 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel 2019-11-07 16:24:11.372 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db 2019-11-07 16:24:11.372 7f873c700dc0 0 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument 2019-11-07 16:24:11.373 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:11.373 7f873c700dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:11.374 7f873c700dc0 0 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:11.374 7f873c700dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal 2019-11-07 16:24:11.374 7f873c700dc0 0 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument 2019-11-07 16:24:11.375 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:11.375 7f873c700dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB 2019-11-07 16:24:11.375 7f873c700dc0 1 bluefs mount 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compaction_readahead_size = 2097152 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compression = kNoCompression 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_background_compactions = 2 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_write_buffer_number = 4 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option recycle_log_file_num = 4 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option writable_file_max_buffer_size = 0 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option write_buffer_size = 268435456 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compaction_readahead_size = 2097152 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compression = kNoCompression 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_background_compactions = 2 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_write_buffer_number = 4 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option recycle_log_file_num = 4 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option writable_file_max_buffer_size = 0 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option write_buffer_size = 268435456 2019-11-07 16:24:11.400 7f873c700dc0 1 rocksdb: do_open column families: [default] 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: RocksDB version: 6.1.2 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Compile date Sep 13 2019 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: DB SUMMARY 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: CURRENT file: CURRENT 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: IDENTITY file: IDENTITY 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: MANIFEST file: MANIFEST-003676 size: 463 Bytes 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: SST files in db dir, Total Num: 5, files: 003598.sst 003600.sst 003602.sst 003604.sst 003606.sst 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Write Ahead Log file in db.wal: 003677.log size: 0 ; 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.error_if_exists: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.create_if_missing: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.paranoid_checks: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.env: 0x55ee2720ea80 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.info_log: 0x55ee2720f220 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_file_opening_threads: 16 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.statistics: (nil) 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_fsync: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_log_file_size: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.log_file_time_to_roll: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.keep_log_file_num: 1000 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.recycle_log_file_num: 4 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_fallocate: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_mmap_reads: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_mmap_writes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_direct_reads: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.create_missing_column_families: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.db_log_dir: 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_dir: db.wal 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.table_cache_numshardbits: 6 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_subcompactions: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_background_flushes: -1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.WAL_ttl_seconds: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.WAL_size_limit_MB: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.is_fd_close_on_exec: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.advise_random_on_open: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.db_write_buffer_size: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_buffer_manager: 0x55ee27526330 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_adaptive_mutex: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.rate_limiter: (nil) 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_recovery_mode: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.enable_thread_tracking: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.enable_pipelined_write: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.row_cache: None 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_filter: None 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_ingest_behind: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.preserve_deletes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.two_write_queues: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.manual_wal_flush: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.atomic_flush: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_background_jobs: 2 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_background_compactions: 2 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.writable_file_max_buffer_size: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.delayed_write_rate : 16777216 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_total_wal_size: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.stats_dump_period_sec: 600 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.stats_persist_period_sec: 600 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_open_files: -1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bytes_per_sync: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_bytes_per_sync: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_readahead_size: 2097152 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Compression algorithms supported: 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kZSTD supported: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kXpressCompression supported: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kLZ4HCCompression supported: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kLZ4Compression supported: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kBZip2Compression supported: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kZlibCompression supported: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kSnappyCompression supported: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Fast CRC32 supported: Supported on x86 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: [db/db_impl_readonly.cc:22] Opening the db in read only mode 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-003676 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_filter: None 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_filter_factory: None 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_factory: SkipListFactory 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.table_factory: BlockBasedTable 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ee264cede0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 1 pin_l0_filter_and_index_blocks_in_cache: 1 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 data_block_hash_table_util_ratio: 0.750000 hash_index_allow_collision: 1 checksum: 1 no_block_cache: 0 block_cache: 0x55ee264e3210 block_cache_name: BinnedLRUCache block_cache_options: capacity : 429496729 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: rocksdb.BuiltinBloomFilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 2 enable_index_compression: 1 block_align: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_buffer_size: 268435456 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_write_buffer_number: 4 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression: NoCompression 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression: Disabled 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.prefix_extractor: nullptr 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.num_levels: 7 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.window_bits: -14 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.level: 32767 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.strategy: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.enabled: false 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.target_file_size_base: 67108864 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.target_file_size_multiplier: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.arena_block_size: 33554432 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.disable_auto_compactions: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.table_properties_collectors: 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.inplace_update_support: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.inplace_update_num_locks: 10000 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_huge_page_size: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bloom_locality: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_successive_merges: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.optimize_filters_for_hits: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.paranoid_file_checks: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.force_consistency_checks: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.report_bg_io_stats: 0 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.ttl: 0 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-003676 succeeded,manifest_file_number is 3676, next_file_number is 3678, last_sequence is 293760899, log_number is 3675,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 3675 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140251439726, "job": 1, "event": "recovery_started", "log_files": [3677]} 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3677 mode 0 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140251439772, "job": 1, "event": "recovery_finished"} 2019-11-07 16:24:11.438 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2 2019-11-07 16:24:11.466 7f873c700dc0 1 freelist init 2019-11-07 16:24:14.027 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_alloc opening allocation metadata 2019-11-07 16:24:14.055 7f873c700dc0 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: In function 'void AllocatorLevel02::_mark_l2_free(int64_t, int64_t) [with L1 = AllocatorLevel01Loose; int64_t = long int]' thread 7f873c700dc0 time 2019-11-07 16:24:14.052212 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: 584: FAILED ceph_assert((int64_t)l2.size() >= (l2_pos_end / d)) ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x55ee1a9e5a24] 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55ee1a9e5bf2] 3: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x55ee1b016870] 4: (BlueStore::_open_alloc()+0x258) [0x55ee1aebcab8] 5: (BlueStore::_open_db_and_around(bool)+0x146) [0x55ee1aede306] 6: (BlueStore::_mount(bool, bool)+0x6a4) [0x55ee1af1cc24] 7: (OSD::init()+0x3aa) [0x55ee1aa8fefa] 8: (main()+0x14fa) [0x55ee1a9ea1da] 9: (__libc_start_main()+0xf5) [0x7f8738372c05] 10: (()+0x4b2695) [0x55ee1aa1f695] 2019-11-07 16:24:14.060 7f873c700dc0 -1 *** Caught signal (Aborted) ** in thread 7f873c700dc0 thread_name:ceph-osd ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (()+0xf5e0) [0x7f87393735e0] 2: (gsignal()+0x37) [0x7f87383861f7] 3: (abort()+0x148) [0x7f87383878e8] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x55ee1a9e5a73] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55ee1a9e5bf2] 6: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x55ee1b016870] 7: (BlueStore::_open_alloc()+0x258) [0x55ee1aebcab8] 8: (BlueStore::_open_db_and_around(bool)+0x146) [0x55ee1aede306] 9: (BlueStore::_mount(bool, bool)+0x6a4) [0x55ee1af1cc24] 10: (OSD::init()+0x3aa) [0x55ee1aa8fefa] 11: (main()+0x14fa) [0x55ee1a9ea1da] 12: (__libc_start_main()+0xf5) [0x7f8738372c05] 13: (()+0x4b2695) [0x55ee1aa1f695] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -549> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command assert hook 0x55ee264d0260 -548> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command abort hook 0x55ee264d0260 -547> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perfcounters_dump hook 0x55ee264d0260 -546> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command 1 hook 0x55ee264d0260 -545> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perf dump hook 0x55ee264d0260 -544> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perfcounters_schema hook 0x55ee264d0260 -543> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perf histogram dump hook 0x55ee264d0260 -542> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command 2 hook 0x55ee264d0260 -541> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perf schema hook 0x55ee264d0260 -540> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perf histogram schema hook 0x55ee264d0260 -539> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command perf reset hook 0x55ee264d0260 -538> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config show hook 0x55ee264d0260 -537> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config help hook 0x55ee264d0260 -536> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config set hook 0x55ee264d0260 -535> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config unset hook 0x55ee264d0260 -534> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config get hook 0x55ee264d0260 -533> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config diff hook 0x55ee264d0260 -532> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command config diff get hook 0x55ee264d0260 -531> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command log flush hook 0x55ee264d0260 -530> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command log dump hook 0x55ee264d0260 -529> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command log reopen hook 0x55ee264d0260 -528> 2019-11-07 16:24:09.960 7f873c700dc0 5 asok(0x55ee26576000) register_command dump_mempools hook 0x55ee2651e7c8 -527> 2019-11-07 16:24:09.968 7f873c700dc0 10 monclient: get_monmap_and_config -526> 2019-11-07 16:24:10.004 7f873c700dc0 10 monclient: build_initial_monmap -525> 2019-11-07 16:24:10.004 7f873c700dc0 10 monclient: monmap: epoch 0 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-11-07 16:24:10.006101 created 2019-11-07 16:24:10.006101 min_mon_release 0 (unknown) 0: v1:131.154.128.145:6789/0 mon.noname-b-legacy 1: v1:131.154.130.69:6789/0 mon.noname-a-legacy 2: v2:131.154.128.145:3300/0 mon.noname-b 3: v2:131.154.130.69:3300/0 mon.noname-a -524> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx -523> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx -522> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx -521> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -520> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -519> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -518> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -517> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -516> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -515> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -514> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -513> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -512> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -511> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -510> 2019-11-07 16:24:10.004 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -509> 2019-11-07 16:24:10.005 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -508> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient: init -507> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding auth protocol: cephx -506> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding auth protocol: cephx -505> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding auth protocol: cephx -504> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: secure -503> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: crc -502> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: secure -501> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: crc -500> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: secure -499> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: crc -498> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: crc -497> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: secure -496> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: crc -495> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: secure -494> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: crc -493> 2019-11-07 16:24:10.006 7f873c700dc0 5 AuthRegistry(0x7ffee49d9d28) adding con mode: secure -492> 2019-11-07 16:24:10.006 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -491> 2019-11-07 16:24:10.006 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -490> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient: _reopen_session rank -1 -489> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient(hunting): picked mon.noname-b-legacy con 0x55ee27136400 addr v1:131.154.128.145:6789/0 -488> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient(hunting): picked mon.noname-a con 0x55ee26573c00 addr v2:131.154.130.69:3300/0 -487> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient(hunting): picked mon.noname-a-legacy con 0x55ee27136c00 addr v1:131.154.130.69:6789/0 -486> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient(hunting): start opening mon connection -485> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient(hunting): _renew_subs -484> 2019-11-07 16:24:10.006 7f873c700dc0 10 monclient(hunting): authenticate will time out at 2019-11-07 16:29:10.007987 -483> 2019-11-07 16:24:10.006 7f873527f700 10 monclient(hunting): get_auth_request con 0x55ee26573c00 auth_method 0 -482> 2019-11-07 16:24:10.006 7f873527f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2] -481> 2019-11-07 16:24:10.006 7f873527f700 10 monclient(hunting): _init_auth method 2 -480> 2019-11-07 16:24:10.007 7f873527f700 10 monclient(hunting): handle_auth_reply_more payload 9 -479> 2019-11-07 16:24:10.007 7f873527f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 -478> 2019-11-07 16:24:10.007 7f873527f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes -477> 2019-11-07 16:24:10.007 7f8734a7e700 10 monclient(hunting): _init_auth method 2 -476> 2019-11-07 16:24:10.007 7f8734a7e700 10 monclient(hunting): my global_id is 491337 -475> 2019-11-07 16:24:10.007 7f873527f700 10 monclient(hunting): handle_auth_done global_id 491332 payload 386 -474> 2019-11-07 16:24:10.007 7f873527f700 10 monclient: _finish_hunting 0 -473> 2019-11-07 16:24:10.007 7f873527f700 1 monclient: found mon.noname-a -472> 2019-11-07 16:24:10.007 7f873527f700 10 monclient: _send_mon_message to mon.noname-a at v2:131.154.130.69:3300/0 -471> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: handle_monmap mon_map magic: 0 v1 -470> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: got monmap 8 from mon.noname-a (according to old e8) -469> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: dump: epoch 8 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-10-08 17:19:02.427513 created 2019-10-03 15:53:10.107437 min_mon_release 14 (nautilus) 0: [v2:131.154.128.131:3300/0,v1:131.154.128.131:6789/0] mon.ds-507 1: [v2:131.154.128.145:3300/0,v1:131.154.128.145:6789/0] mon.ds-001 2: [v2:131.154.130.69:3300/0,v1:131.154.130.69:6789/0] mon.qn-cnfslhc 3: [v2:131.154.130.77:3300/0,v1:131.154.130.77:6789/0] mon.ds-303 4: [v2:131.154.130.166:3300/0,v1:131.154.130.166:6789/0] mon.cs-001 -468> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: _finish_auth 0 -467> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-11-07 16:23:40.009983) -466> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: _send_mon_message to mon.qn-cnfslhc at v2:131.154.130.69:3300/0 -465> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: handle_config config(1 keys) v1 -464> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: handle_monmap mon_map magic: 0 v1 -463> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: got monmap 8 from mon.qn-cnfslhc (according to old e8) -462> 2019-11-07 16:24:10.008 7f8734a7e700 10 monclient: dump: epoch 8 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-10-08 17:19:02.427513 created 2019-10-03 15:53:10.107437 min_mon_release 14 (nautilus) 0: [v2:131.154.128.131:3300/0,v1:131.154.128.131:6789/0] mon.ds-507 1: [v2:131.154.128.145:3300/0,v1:131.154.128.145:6789/0] mon.ds-001 2: [v2:131.154.130.69:3300/0,v1:131.154.130.69:6789/0] mon.qn-cnfslhc 3: [v2:131.154.130.77:3300/0,v1:131.154.130.77:6789/0] mon.ds-303 4: [v2:131.154.130.166:3300/0,v1:131.154.130.166:6789/0] mon.cs-001 -461> 2019-11-07 16:24:10.008 7f873327b700 4 set_mon_vals no callback set -460> 2019-11-07 16:24:10.008 7f873c700dc0 5 monclient: authenticate success, global_id 491332 -459> 2019-11-07 16:24:10.008 7f873c700dc0 10 monclient: get_monmap_and_config success -458> 2019-11-07 16:24:10.008 7f873c700dc0 10 monclient: shutdown -457> 2019-11-07 16:24:10.008 7f873327b700 10 set_mon_vals debug_auth = 20/20 -456> 2019-11-07 16:24:10.011 7f873c700dc0 0 set uid:gid to 167:167 (ceph:ceph) -455> 2019-11-07 16:24:10.011 7f873c700dc0 0 ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable), process ceph-osd, pid 2350481 -454> 2019-11-07 16:24:10.011 7f873c700dc0 0 pidfile_write: ignore empty --pid-file -453> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) init /var/run/ceph/ceph-osd.20.asok -452> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) bind_and_listen /var/run/ceph/ceph-osd.20.asok -451> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) register_command 0 hook 0x55ee264cf648 -450> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) register_command version hook 0x55ee264cf648 -449> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) register_command git_version hook 0x55ee264cf648 -448> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) register_command help hook 0x55ee264d02d0 -447> 2019-11-07 16:24:10.046 7f873c700dc0 5 asok(0x55ee26576000) register_command get_command_descriptions hook 0x55ee264d0320 -446> 2019-11-07 16:24:10.046 7f8733a7c700 5 asok(0x55ee26576000) entry start -445> 2019-11-07 16:24:10.047 7f873c700dc0 5 object store type is bluestore -444> 2019-11-07 16:24:10.048 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -443> 2019-11-07 16:24:10.048 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -442> 2019-11-07 16:24:10.048 7f873c700dc0 0 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -441> 2019-11-07 16:24:10.049 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -440> 2019-11-07 16:24:10.049 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -439> 2019-11-07 16:24:10.049 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel -438> 2019-11-07 16:24:10.049 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db -437> 2019-11-07 16:24:10.049 7f873c700dc0 0 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument -436> 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -435> 2019-11-07 16:24:10.050 7f873c700dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB -434> 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -433> 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -432> 2019-11-07 16:24:10.050 7f873c700dc0 0 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -431> 2019-11-07 16:24:10.050 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -430> 2019-11-07 16:24:10.050 7f873c700dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB -429> 2019-11-07 16:24:10.051 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel -428> 2019-11-07 16:24:10.051 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal -427> 2019-11-07 16:24:10.051 7f873c700dc0 0 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument -426> 2019-11-07 16:24:10.051 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -425> 2019-11-07 16:24:10.051 7f873c700dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB -424> 2019-11-07 16:24:10.054 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.wal) close -423> 2019-11-07 16:24:10.308 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block.db) close -422> 2019-11-07 16:24:10.555 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block) close -421> 2019-11-07 16:24:10.809 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) close -420> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding auth protocol: cephx -419> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_method_list cephx -> [2] -418> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding auth protocol: cephx -417> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_method_list cephx -> [2] -416> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding auth protocol: cephx -415> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_method_list cephx -> [2] -414> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure -413> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc -412> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list secure crc -> [2,1] -411> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure -410> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc -409> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list secure crc -> [2,1] -408> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure -407> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc -406> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list secure crc -> [2,1] -405> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc -404> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure -403> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list crc secure -> [1,2] -402> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc -401> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure -400> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list crc secure -> [1,2] -399> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: crc -398> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553338) adding con mode: secure -397> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553338) _parse_mode_list crc secure -> [1,2] -396> 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553338) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -395> 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553338) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -394> 2019-11-07 16:24:11.063 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -393> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx -392> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_method_list cephx -> [2] -391> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx -390> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_method_list cephx -> [2] -389> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding auth protocol: cephx -388> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_method_list cephx -> [2] -387> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -386> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -385> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list secure crc -> [2,1] -384> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -383> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -382> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list secure crc -> [2,1] -381> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -380> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -379> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list secure crc -> [2,1] -378> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -377> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -376> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list crc secure -> [1,2] -375> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -374> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -373> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list crc secure -> [1,2] -372> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: crc -371> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26552a38) adding con mode: secure -370> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26552a38) _parse_mode_list crc secure -> [1,2] -369> 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26552a38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -368> 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26552a38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -367> 2019-11-07 16:24:11.063 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -366> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding auth protocol: cephx -365> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_method_list cephx -> [2] -364> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding auth protocol: cephx -363> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_method_list cephx -> [2] -362> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding auth protocol: cephx -361> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_method_list cephx -> [2] -360> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure -359> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc -358> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list secure crc -> [2,1] -357> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure -356> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc -355> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list secure crc -> [2,1] -354> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure -353> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc -352> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list secure crc -> [2,1] -351> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc -350> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure -349> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list crc secure -> [1,2] -348> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc -347> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure -346> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list crc secure -> [1,2] -345> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: crc -344> 2019-11-07 16:24:11.063 7f873c700dc0 5 AuthRegistry(0x55ee26553c38) adding con mode: secure -343> 2019-11-07 16:24:11.063 7f873c700dc0 20 AuthRegistry(0x55ee26553c38) _parse_mode_list crc secure -> [1,2] -342> 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553c38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -341> 2019-11-07 16:24:11.063 7f873c700dc0 10 AuthRegistry(0x55ee26553c38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -340> 2019-11-07 16:24:11.063 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -339> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding auth protocol: cephx -338> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_method_list cephx -> [2] -337> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding auth protocol: cephx -336> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_method_list cephx -> [2] -335> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding auth protocol: cephx -334> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_method_list cephx -> [2] -333> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure -332> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc -331> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list secure crc -> [2,1] -330> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure -329> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc -328> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list secure crc -> [2,1] -327> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure -326> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc -325> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list secure crc -> [2,1] -324> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc -323> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure -322> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list crc secure -> [1,2] -321> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc -320> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure -319> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list crc secure -> [1,2] -318> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: crc -317> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554538) adding con mode: secure -316> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554538) _parse_mode_list crc secure -> [1,2] -315> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554538) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -314> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554538) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -313> 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -312> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding auth protocol: cephx -311> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_method_list cephx -> [2] -310> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding auth protocol: cephx -309> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_method_list cephx -> [2] -308> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding auth protocol: cephx -307> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_method_list cephx -> [2] -306> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure -305> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc -304> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list secure crc -> [2,1] -303> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure -302> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc -301> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list secure crc -> [2,1] -300> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure -299> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc -298> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list secure crc -> [2,1] -297> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc -296> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure -295> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list crc secure -> [1,2] -294> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc -293> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure -292> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list crc secure -> [1,2] -291> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: crc -290> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26554e38) adding con mode: secure -289> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26554e38) _parse_mode_list crc secure -> [1,2] -288> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554e38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -287> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26554e38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -286> 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -285> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding auth protocol: cephx -284> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_method_list cephx -> [2] -283> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding auth protocol: cephx -282> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_method_list cephx -> [2] -281> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding auth protocol: cephx -280> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_method_list cephx -> [2] -279> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure -278> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc -277> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list secure crc -> [2,1] -276> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure -275> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc -274> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list secure crc -> [2,1] -273> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure -272> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc -271> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list secure crc -> [2,1] -270> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc -269> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure -268> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list crc secure -> [1,2] -267> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc -266> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure -265> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list crc secure -> [1,2] -264> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: crc -263> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee26555738) adding con mode: secure -262> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee26555738) _parse_mode_list crc secure -> [1,2] -261> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26555738) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -260> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee26555738) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -259> 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -258> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding auth protocol: cephx -257> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_method_list cephx -> [2] -256> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding auth protocol: cephx -255> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_method_list cephx -> [2] -254> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding auth protocol: cephx -253> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_method_list cephx -> [2] -252> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure -251> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc -250> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list secure crc -> [2,1] -249> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure -248> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc -247> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list secure crc -> [2,1] -246> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure -245> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc -244> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list secure crc -> [2,1] -243> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc -242> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure -241> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list crc secure -> [1,2] -240> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc -239> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure -238> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list crc secure -> [1,2] -237> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: crc -236> 2019-11-07 16:24:11.064 7f873c700dc0 5 AuthRegistry(0x55ee2724e138) adding con mode: secure -235> 2019-11-07 16:24:11.064 7f873c700dc0 20 AuthRegistry(0x55ee2724e138) _parse_mode_list crc secure -> [1,2] -234> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee2724e138) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -233> 2019-11-07 16:24:11.064 7f873c700dc0 10 AuthRegistry(0x55ee2724e138) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -232> 2019-11-07 16:24:11.064 7f873c700dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -231> 2019-11-07 16:24:11.064 7f873c700dc0 0 starting osd.20 osd_data /var/lib/ceph/osd/ceph-20 /var/lib/ceph/osd/ceph-20/journal -230> 2019-11-07 16:24:11.064 7f873c700dc0 -1 Falling back to public interface -229> 2019-11-07 16:24:11.094 7f873c700dc0 10 monclient: build_initial_monmap -228> 2019-11-07 16:24:11.094 7f873c700dc0 10 monclient: monmap: epoch 0 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-11-07 16:24:11.095949 created 2019-11-07 16:24:11.095949 min_mon_release 0 (unknown) 0: v1:131.154.128.145:6789/0 mon.noname-b-legacy 1: v1:131.154.130.69:6789/0 mon.noname-a-legacy 2: v2:131.154.128.145:3300/0 mon.noname-b 3: v2:131.154.130.69:3300/0 mon.noname-a -227> 2019-11-07 16:24:11.105 7f873c700dc0 0 load: jerasure load: lrc load: isa -226> 2019-11-07 16:24:11.105 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -225> 2019-11-07 16:24:11.105 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -224> 2019-11-07 16:24:11.106 7f873c700dc0 0 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -223> 2019-11-07 16:24:11.106 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -222> 2019-11-07 16:24:11.106 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -221> 2019-11-07 16:24:11.106 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) close -220> 2019-11-07 16:24:11.370 7f873c700dc0 5 asok(0x55ee26576000) register_command objecter_requests hook 0x55ee264d0a60 -219> 2019-11-07 16:24:11.371 7f873c700dc0 2 osd.20 0 init /var/lib/ceph/osd/ceph-20 (looks like hdd) -218> 2019-11-07 16:24:11.371 7f873c700dc0 2 osd.20 0 journal /var/lib/ceph/osd/ceph-20/journal -217> 2019-11-07 16:24:11.371 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _mount path /var/lib/ceph/osd/ceph-20 -216> 2019-11-07 16:24:11.371 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -215> 2019-11-07 16:24:11.371 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -214> 2019-11-07 16:24:11.371 7f873c700dc0 0 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -213> 2019-11-07 16:24:11.372 7f873c700dc0 1 bdev(0x55ee27114000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -212> 2019-11-07 16:24:11.372 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -211> 2019-11-07 16:24:11.372 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel -210> 2019-11-07 16:24:11.372 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db -209> 2019-11-07 16:24:11.372 7f873c700dc0 0 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument -208> 2019-11-07 16:24:11.373 7f873c700dc0 1 bdev(0x55ee27114e00 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -207> 2019-11-07 16:24:11.373 7f873c700dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB -206> 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -205> 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -204> 2019-11-07 16:24:11.374 7f873c700dc0 0 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -203> 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev(0x55ee27114700 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -202> 2019-11-07 16:24:11.374 7f873c700dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB -201> 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel -200> 2019-11-07 16:24:11.374 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal -199> 2019-11-07 16:24:11.374 7f873c700dc0 0 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument -198> 2019-11-07 16:24:11.375 7f873c700dc0 1 bdev(0x55ee27114a80 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -197> 2019-11-07 16:24:11.375 7f873c700dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB -196> 2019-11-07 16:24:11.375 7f873c700dc0 1 bluefs mount -195> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compaction_readahead_size = 2097152 -194> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compression = kNoCompression -193> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_background_compactions = 2 -192> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_write_buffer_number = 4 -191> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 -190> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option recycle_log_file_num = 4 -189> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option writable_file_max_buffer_size = 0 -188> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option write_buffer_size = 268435456 -187> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compaction_readahead_size = 2097152 -186> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option compression = kNoCompression -185> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_background_compactions = 2 -184> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option max_write_buffer_number = 4 -183> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 -182> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option recycle_log_file_num = 4 -181> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option writable_file_max_buffer_size = 0 -180> 2019-11-07 16:24:11.395 7f873c700dc0 0 set rocksdb option write_buffer_size = 268435456 -179> 2019-11-07 16:24:11.400 7f873c700dc0 1 rocksdb: do_open column families: [default] -178> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: RocksDB version: 6.1.2 -177> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ -176> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Compile date Sep 13 2019 -175> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: DB SUMMARY -174> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: CURRENT file: CURRENT -173> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: IDENTITY file: IDENTITY -172> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: MANIFEST file: MANIFEST-003676 size: 463 Bytes -171> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: SST files in db dir, Total Num: 5, files: 003598.sst 003600.sst 003602.sst 003604.sst 003606.sst -170> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: -169> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Write Ahead Log file in db.wal: 003677.log size: 0 ; -168> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.error_if_exists: 0 -167> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.create_if_missing: 0 -166> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.paranoid_checks: 1 -165> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.env: 0x55ee2720ea80 -164> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.info_log: 0x55ee2720f220 -163> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_file_opening_threads: 16 -162> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.statistics: (nil) -161> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_fsync: 0 -160> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_log_file_size: 0 -159> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_manifest_file_size: 1073741824 -158> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.log_file_time_to_roll: 0 -157> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.keep_log_file_num: 1000 -156> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.recycle_log_file_num: 4 -155> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_fallocate: 1 -154> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_mmap_reads: 0 -153> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_mmap_writes: 0 -152> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_direct_reads: 0 -151> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 -150> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.create_missing_column_families: 0 -149> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.db_log_dir: -148> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_dir: db.wal -147> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.table_cache_numshardbits: 6 -146> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_subcompactions: 1 -145> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_background_flushes: -1 -144> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.WAL_ttl_seconds: 0 -143> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.WAL_size_limit_MB: 0 -142> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.manifest_preallocation_size: 4194304 -141> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.is_fd_close_on_exec: 1 -140> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.advise_random_on_open: 1 -139> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.db_write_buffer_size: 0 -138> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_buffer_manager: 0x55ee27526330 -137> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.access_hint_on_compaction_start: 1 -136> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1 -135> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 -134> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.use_adaptive_mutex: 0 -133> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.rate_limiter: (nil) -132> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 -131> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_recovery_mode: 0 -130> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.enable_thread_tracking: 0 -129> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.enable_pipelined_write: 0 -128> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 -127> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 -126> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_thread_max_yield_usec: 100 -125> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 -124> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.row_cache: None -123> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_filter: None -122> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.avoid_flush_during_recovery: 0 -121> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.allow_ingest_behind: 0 -120> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.preserve_deletes: 0 -119> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.two_write_queues: 0 -118> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.manual_wal_flush: 0 -117> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.atomic_flush: 0 -116> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 -115> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_background_jobs: 2 -114> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_background_compactions: 2 -113> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 -112> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.writable_file_max_buffer_size: 0 -111> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.delayed_write_rate : 16777216 -110> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_total_wal_size: 0 -109> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 -108> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.stats_dump_period_sec: 600 -107> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.stats_persist_period_sec: 600 -106> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.stats_history_buffer_size: 1048576 -105> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_open_files: -1 -104> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bytes_per_sync: 0 -103> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.wal_bytes_per_sync: 0 -102> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_readahead_size: 2097152 -101> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Compression algorithms supported: -100> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kZSTDNotFinalCompression supported: 0 -99> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kZSTD supported: 0 -98> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kXpressCompression supported: 0 -97> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kLZ4HCCompression supported: 1 -96> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kLZ4Compression supported: 1 -95> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kBZip2Compression supported: 0 -94> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kZlibCompression supported: 1 -93> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: kSnappyCompression supported: 1 -92> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Fast CRC32 supported: Supported on x86 -91> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: [db/db_impl_readonly.cc:22] Opening the db in read only mode -90> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-003676 -89> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: -88> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator -87> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor -86> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_filter: None -85> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_filter_factory: None -84> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_factory: SkipListFactory -83> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.table_factory: BlockBasedTable -82> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ee264cede0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 1 pin_l0_filter_and_index_blocks_in_cache: 1 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 data_block_hash_table_util_ratio: 0.750000 hash_index_allow_collision: 1 checksum: 1 no_block_cache: 0 block_cache: 0x55ee264e3210 block_cache_name: BinnedLRUCache block_cache_options: capacity : 429496729 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: rocksdb.BuiltinBloomFilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 2 enable_index_compression: 1 block_align: 0 -81> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.write_buffer_size: 268435456 -80> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_write_buffer_number: 4 -79> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression: NoCompression -78> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression: Disabled -77> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.prefix_extractor: nullptr -76> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr -75> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.num_levels: 7 -74> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 -73> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 -72> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 -71> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 -70> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 -69> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 -68> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 -67> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bottommost_compression_opts.enabled: false -66> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.window_bits: -14 -65> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.level: 32767 -64> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.strategy: 0 -63> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 -62> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 -61> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compression_opts.enabled: false -60> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 -59> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 -58> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level0_stop_writes_trigger: 36 -57> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.target_file_size_base: 67108864 -56> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.target_file_size_multiplier: 1 -55> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 -54> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 -53> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 -52> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 -51> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 -50> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 -49> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 -48> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 -47> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 -46> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 -45> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 -44> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_compaction_bytes: 1677721600 -43> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.arena_block_size: 33554432 -42> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 -41> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 -40> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 -39> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.disable_auto_compactions: 0 -38> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel -37> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio -36> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 -35> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 -34> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 -33> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 -32> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 -31> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize -30> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 -29> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 -28> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.table_properties_collectors: -27> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.inplace_update_support: 0 -26> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.inplace_update_num_locks: 10000 -25> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 -24> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_whole_key_filtering: 0 -23> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.memtable_huge_page_size: 0 -22> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.bloom_locality: 0 -21> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.max_successive_merges: 0 -20> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.optimize_filters_for_hits: 0 -19> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.paranoid_file_checks: 0 -18> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.force_consistency_checks: 0 -17> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.report_bg_io_stats: 0 -16> 2019-11-07 16:24:11.401 7f873c700dc0 4 rocksdb: Options.ttl: 0 -15> 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-003676 succeeded,manifest_file_number is 3676, next_file_number is 3678, last_sequence is 293760899, log_number is 3675,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 -14> 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 3675 -13> 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140251439726, "job": 1, "event": "recovery_started", "log_files": [3677]} -12> 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3677 mode 0 -11> 2019-11-07 16:24:11.438 7f873c700dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140251439772, "job": 1, "event": "recovery_finished"} -10> 2019-11-07 16:24:11.438 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2 -9> 2019-11-07 16:24:11.465 7f873c700dc0 1 get compressor snappy = 0 -8> 2019-11-07 16:24:11.465 7f873c700dc0 1 load compressor snappy -7> 2019-11-07 16:24:11.465 7f873c700dc0 1 add compressor snappy 0x55ee27526960 -6> 2019-11-07 16:24:11.466 7f873c700dc0 1 get compressor snappy = 0x55ee27526960 -5> 2019-11-07 16:24:11.466 7f873c700dc0 1 load: compressor snappy loaded and registered -4> 2019-11-07 16:24:11.466 7f873c700dc0 1 get compressor snappy = 0x55ee27526960 -3> 2019-11-07 16:24:11.466 7f873c700dc0 1 freelist init -2> 2019-11-07 16:24:14.027 7f873c700dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_alloc opening allocation metadata -1> 2019-11-07 16:24:14.055 7f873c700dc0 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: In function 'void AllocatorLevel02::_mark_l2_free(int64_t, int64_t) [with L1 = AllocatorLevel01Loose; int64_t = long int]' thread 7f873c700dc0 time 2019-11-07 16:24:14.052212 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: 584: FAILED ceph_assert((int64_t)l2.size() >= (l2_pos_end / d)) ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x55ee1a9e5a24] 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55ee1a9e5bf2] 3: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x55ee1b016870] 4: (BlueStore::_open_alloc()+0x258) [0x55ee1aebcab8] 5: (BlueStore::_open_db_and_around(bool)+0x146) [0x55ee1aede306] 6: (BlueStore::_mount(bool, bool)+0x6a4) [0x55ee1af1cc24] 7: (OSD::init()+0x3aa) [0x55ee1aa8fefa] 8: (main()+0x14fa) [0x55ee1a9ea1da] 9: (__libc_start_main()+0xf5) [0x7f8738372c05] 10: (()+0x4b2695) [0x55ee1aa1f695] 0> 2019-11-07 16:24:14.060 7f873c700dc0 -1 *** Caught signal (Aborted) ** in thread 7f873c700dc0 thread_name:ceph-osd ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (()+0xf5e0) [0x7f87393735e0] 2: (gsignal()+0x37) [0x7f87383861f7] 3: (abort()+0x148) [0x7f87383878e8] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x55ee1a9e5a73] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55ee1a9e5bf2] 6: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x55ee1b016870] 7: (BlueStore::_open_alloc()+0x258) [0x55ee1aebcab8] 8: (BlueStore::_open_db_and_around(bool)+0x146) [0x55ee1aede306] 9: (BlueStore::_mount(bool, bool)+0x6a4) [0x55ee1af1cc24] 10: (OSD::init()+0x3aa) [0x55ee1aa8fefa] 11: (main()+0x14fa) [0x55ee1a9ea1da] 12: (__libc_start_main()+0xf5) [0x7f8738372c05] 13: (()+0x4b2695) [0x55ee1aa1f695] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 0 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 20/20 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 rgw_sync 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.20.log --- end dump of recent events --- 2019-11-07 16:24:14.779 7f811f851dc0 0 set uid:gid to 167:167 (ceph:ceph) 2019-11-07 16:24:14.779 7f811f851dc0 0 ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable), process ceph-osd, pid 2350606 2019-11-07 16:24:14.779 7f811f851dc0 0 pidfile_write: ignore empty --pid-file 2019-11-07 16:24:14.816 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:14.816 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:14.816 7f811f851dc0 0 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:14.817 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:14.817 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-11-07 16:24:14.817 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel 2019-11-07 16:24:14.817 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db 2019-11-07 16:24:14.817 7f811f851dc0 0 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument 2019-11-07 16:24:14.818 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:14.818 7f811f851dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB 2019-11-07 16:24:14.818 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:14.818 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:14.819 7f811f851dc0 0 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:14.819 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:14.819 7f811f851dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB 2019-11-07 16:24:14.819 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel 2019-11-07 16:24:14.819 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal 2019-11-07 16:24:14.819 7f811f851dc0 0 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument 2019-11-07 16:24:14.820 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:14.820 7f811f851dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB 2019-11-07 16:24:14.821 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) close 2019-11-07 16:24:15.078 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) close 2019-11-07 16:24:15.323 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) close 2019-11-07 16:24:15.577 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) close 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a964a38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a964a38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding auth protocol: cephx 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding auth protocol: cephx 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding auth protocol: cephx 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a965338) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a965338) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.824 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a965c38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a965c38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966538) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966538) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966e38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966e38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a967738) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a967738) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding auth protocol: cephx 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_method_list cephx -> [2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list secure crc -> [2,1] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list crc secure -> [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038b676138) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038b676138) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring 2019-11-07 16:24:15.825 7f811f851dc0 0 starting osd.20 osd_data /var/lib/ceph/osd/ceph-20 /var/lib/ceph/osd/ceph-20/journal 2019-11-07 16:24:15.825 7f811f851dc0 -1 Falling back to public interface 2019-11-07 16:24:15.869 7f811f851dc0 0 load: jerasure load: lrc load: isa 2019-11-07 16:24:15.869 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:15.869 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:15.869 7f811f851dc0 0 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:15.869 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:15.870 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-11-07 16:24:15.870 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) close 2019-11-07 16:24:16.132 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _mount path /var/lib/ceph/osd/ceph-20 2019-11-07 16:24:16.132 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:16.132 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:16.132 7f811f851dc0 0 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:16.133 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:16.133 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 2019-11-07 16:24:16.133 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel 2019-11-07 16:24:16.133 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db 2019-11-07 16:24:16.133 7f811f851dc0 0 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument 2019-11-07 16:24:16.134 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:16.134 7f811f851dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB 2019-11-07 16:24:16.135 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel 2019-11-07 16:24:16.135 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block 2019-11-07 16:24:16.135 7f811f851dc0 0 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported 2019-11-07 16:24:16.136 7f811f851dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal 2019-11-07 16:24:16.136 7f811f851dc0 0 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported 2019-11-07 16:24:16.136 7f811f851dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB 2019-11-07 16:24:16.137 7f811f851dc0 1 bluefs mount 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compaction_readahead_size = 2097152 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compression = kNoCompression 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_background_compactions = 2 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_write_buffer_number = 4 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option recycle_log_file_num = 4 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option writable_file_max_buffer_size = 0 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option write_buffer_size = 268435456 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compaction_readahead_size = 2097152 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compression = kNoCompression 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_background_compactions = 2 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_write_buffer_number = 4 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option recycle_log_file_num = 4 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option writable_file_max_buffer_size = 0 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option write_buffer_size = 268435456 2019-11-07 16:24:16.170 7f811f851dc0 1 rocksdb: do_open column families: [default] 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: RocksDB version: 6.1.2 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Compile date Sep 13 2019 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: DB SUMMARY 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: CURRENT file: CURRENT 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: IDENTITY file: IDENTITY 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: MANIFEST file: MANIFEST-003676 size: 463 Bytes 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: SST files in db dir, Total Num: 5, files: 003598.sst 003600.sst 003602.sst 003604.sst 003606.sst 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Write Ahead Log file in db.wal: 003677.log size: 0 ; 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.error_if_exists: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.create_if_missing: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.paranoid_checks: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.env: 0x56038b624a60 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.info_log: 0x56038b625200 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_file_opening_threads: 16 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.statistics: (nil) 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_fsync: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_log_file_size: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.log_file_time_to_roll: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.keep_log_file_num: 1000 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.recycle_log_file_num: 4 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_fallocate: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_mmap_reads: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_mmap_writes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_direct_reads: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.create_missing_column_families: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.db_log_dir: 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_dir: db.wal 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.table_cache_numshardbits: 6 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_subcompactions: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_background_flushes: -1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.WAL_ttl_seconds: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.WAL_size_limit_MB: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.is_fd_close_on_exec: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.advise_random_on_open: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.db_write_buffer_size: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_buffer_manager: 0x56038b940360 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_adaptive_mutex: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.rate_limiter: (nil) 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_recovery_mode: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.enable_thread_tracking: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.enable_pipelined_write: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.row_cache: None 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_filter: None 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_ingest_behind: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.preserve_deletes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.two_write_queues: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.manual_wal_flush: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.atomic_flush: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_background_jobs: 2 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_background_compactions: 2 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.writable_file_max_buffer_size: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.delayed_write_rate : 16777216 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_total_wal_size: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.stats_dump_period_sec: 600 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.stats_persist_period_sec: 600 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_open_files: -1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bytes_per_sync: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_bytes_per_sync: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_readahead_size: 2097152 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Compression algorithms supported: 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kZSTD supported: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kXpressCompression supported: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kLZ4HCCompression supported: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kLZ4Compression supported: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kBZip2Compression supported: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kZlibCompression supported: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kSnappyCompression supported: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Fast CRC32 supported: Supported on x86 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: [db/db_impl_readonly.cc:22] Opening the db in read only mode 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-003676 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_filter: None 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_filter_factory: None 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_factory: SkipListFactory 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.table_factory: BlockBasedTable 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56038a8e0de0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 1 pin_l0_filter_and_index_blocks_in_cache: 1 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 data_block_hash_table_util_ratio: 0.750000 hash_index_allow_collision: 1 checksum: 1 no_block_cache: 0 block_cache: 0x56038a8f5210 block_cache_name: BinnedLRUCache block_cache_options: capacity : 429496729 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: rocksdb.BuiltinBloomFilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 2 enable_index_compression: 1 block_align: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_buffer_size: 268435456 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_write_buffer_number: 4 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression: NoCompression 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression: Disabled 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.prefix_extractor: nullptr 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.num_levels: 7 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.window_bits: -14 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.level: 32767 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.strategy: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.enabled: false 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.target_file_size_base: 67108864 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.target_file_size_multiplier: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.arena_block_size: 33554432 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.disable_auto_compactions: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.table_properties_collectors: 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.inplace_update_support: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.inplace_update_num_locks: 10000 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_huge_page_size: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bloom_locality: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_successive_merges: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.optimize_filters_for_hits: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.paranoid_file_checks: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.force_consistency_checks: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.report_bg_io_stats: 0 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.ttl: 0 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-003676 succeeded,manifest_file_number is 3676, next_file_number is 3678, last_sequence is 293760899, log_number is 3675,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 3675 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140256220733, "job": 1, "event": "recovery_started", "log_files": [3677]} 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3677 mode 0 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140256220762, "job": 1, "event": "recovery_finished"} 2019-11-07 16:24:16.219 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2 2019-11-07 16:24:16.243 7f811f851dc0 1 freelist init 2019-11-07 16:24:17.621 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_alloc opening allocation metadata 2019-11-07 16:24:17.684 7f811f851dc0 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: In function 'void AllocatorLevel02::_mark_l2_free(int64_t, int64_t) [with L1 = AllocatorLevel01Loose; int64_t = long int]' thread 7f811f851dc0 time 2019-11-07 16:24:17.681601 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: 584: FAILED ceph_assert((int64_t)l2.size() >= (l2_pos_end / d)) ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x56037f236a24] 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x56037f236bf2] 3: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x56037f867870] 4: (BlueStore::_open_alloc()+0x258) [0x56037f70dab8] 5: (BlueStore::_open_db_and_around(bool)+0x146) [0x56037f72f306] 6: (BlueStore::_mount(bool, bool)+0x6a4) [0x56037f76dc24] 7: (OSD::init()+0x3aa) [0x56037f2e0efa] 8: (main()+0x14fa) [0x56037f23b1da] 9: (__libc_start_main()+0xf5) [0x7f811b4c3c05] 10: (()+0x4b2695) [0x56037f270695] 2019-11-07 16:24:17.689 7f811f851dc0 -1 *** Caught signal (Aborted) ** in thread 7f811f851dc0 thread_name:ceph-osd ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (()+0xf5e0) [0x7f811c4c45e0] 2: (gsignal()+0x37) [0x7f811b4d71f7] 3: (abort()+0x148) [0x7f811b4d88e8] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x56037f236a73] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x56037f236bf2] 6: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x56037f867870] 7: (BlueStore::_open_alloc()+0x258) [0x56037f70dab8] 8: (BlueStore::_open_db_and_around(bool)+0x146) [0x56037f72f306] 9: (BlueStore::_mount(bool, bool)+0x6a4) [0x56037f76dc24] 10: (OSD::init()+0x3aa) [0x56037f2e0efa] 11: (main()+0x14fa) [0x56037f23b1da] 12: (__libc_start_main()+0xf5) [0x7f811b4c3c05] 13: (()+0x4b2695) [0x56037f270695] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -551> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command assert hook 0x56038a8e2260 -550> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command abort hook 0x56038a8e2260 -549> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perfcounters_dump hook 0x56038a8e2260 -548> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command 1 hook 0x56038a8e2260 -547> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perf dump hook 0x56038a8e2260 -546> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perfcounters_schema hook 0x56038a8e2260 -545> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perf histogram dump hook 0x56038a8e2260 -544> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command 2 hook 0x56038a8e2260 -543> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perf schema hook 0x56038a8e2260 -542> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perf histogram schema hook 0x56038a8e2260 -541> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command perf reset hook 0x56038a8e2260 -540> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config show hook 0x56038a8e2260 -539> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config help hook 0x56038a8e2260 -538> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config set hook 0x56038a8e2260 -537> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config unset hook 0x56038a8e2260 -536> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config get hook 0x56038a8e2260 -535> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config diff hook 0x56038a8e2260 -534> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command config diff get hook 0x56038a8e2260 -533> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command log flush hook 0x56038a8e2260 -532> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command log dump hook 0x56038a8e2260 -531> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command log reopen hook 0x56038a8e2260 -530> 2019-11-07 16:24:14.727 7f811f851dc0 5 asok(0x56038a988000) register_command dump_mempools hook 0x56038a9307c8 -529> 2019-11-07 16:24:14.736 7f811f851dc0 10 monclient: get_monmap_and_config -528> 2019-11-07 16:24:14.772 7f811f851dc0 10 monclient: build_initial_monmap -527> 2019-11-07 16:24:14.773 7f811f851dc0 10 monclient: monmap: epoch 0 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-11-07 16:24:14.774473 created 2019-11-07 16:24:14.774473 min_mon_release 0 (unknown) 0: v1:131.154.128.145:6789/0 mon.noname-b-legacy 1: v1:131.154.130.69:6789/0 mon.noname-a-legacy 2: v2:131.154.128.145:3300/0 mon.noname-b 3: v2:131.154.130.69:3300/0 mon.noname-a -526> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx -525> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx -524> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx -523> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -522> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -521> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -520> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -519> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -518> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -517> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -516> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -515> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -514> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -513> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -512> 2019-11-07 16:24:14.773 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -511> 2019-11-07 16:24:14.773 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -510> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient: init -509> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding auth protocol: cephx -508> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding auth protocol: cephx -507> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding auth protocol: cephx -506> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: secure -505> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: crc -504> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: secure -503> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: crc -502> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: secure -501> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: crc -500> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: crc -499> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: secure -498> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: crc -497> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: secure -496> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: crc -495> 2019-11-07 16:24:14.774 7f811f851dc0 5 AuthRegistry(0x7ffc8a842268) adding con mode: secure -494> 2019-11-07 16:24:14.774 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -493> 2019-11-07 16:24:14.774 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -492> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient: _reopen_session rank -1 -491> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): picked mon.noname-a con 0x56038b548400 addr v2:131.154.130.69:3300/0 -490> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): picked mon.noname-b-legacy con 0x56038a985c00 addr v1:131.154.128.145:6789/0 -489> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): picked mon.noname-b con 0x56038b548c00 addr v2:131.154.128.145:3300/0 -488> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): start opening mon connection -487> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): start opening mon connection -486> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): _renew_subs -485> 2019-11-07 16:24:14.774 7f811f851dc0 10 monclient(hunting): authenticate will time out at 2019-11-07 16:29:14.776363 -484> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): get_auth_request con 0x56038b548400 auth_method 0 -483> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2] -482> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): _init_auth method 2 -481> 2019-11-07 16:24:14.775 7f81193d2700 10 monclient(hunting): get_auth_request con 0x56038b548c00 auth_method 0 -480> 2019-11-07 16:24:14.775 7f81193d2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2] -479> 2019-11-07 16:24:14.775 7f81193d2700 10 monclient(hunting): _init_auth method 2 -478> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): handle_auth_reply_more payload 9 -477> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 -476> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes -475> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient(hunting): handle_auth_done global_id 491342 payload 386 -474> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient: _finish_hunting 0 -473> 2019-11-07 16:24:14.775 7f8118bd1700 1 monclient: found mon.noname-a -472> 2019-11-07 16:24:14.775 7f8118bd1700 10 monclient: _send_mon_message to mon.noname-a at v2:131.154.130.69:3300/0 -471> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: handle_monmap mon_map magic: 0 v1 -470> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: got monmap 8 from mon.noname-a (according to old e8) -469> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: dump: epoch 8 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-10-08 17:19:02.427513 created 2019-10-03 15:53:10.107437 min_mon_release 14 (nautilus) 0: [v2:131.154.128.131:3300/0,v1:131.154.128.131:6789/0] mon.ds-507 1: [v2:131.154.128.145:3300/0,v1:131.154.128.145:6789/0] mon.ds-001 2: [v2:131.154.130.69:3300/0,v1:131.154.130.69:6789/0] mon.qn-cnfslhc 3: [v2:131.154.130.77:3300/0,v1:131.154.130.77:6789/0] mon.ds-303 4: [v2:131.154.130.166:3300/0,v1:131.154.130.166:6789/0] mon.cs-001 -468> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: _finish_auth 0 -467> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-11-07 16:23:44.778220) -466> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: _send_mon_message to mon.qn-cnfslhc at v2:131.154.130.69:3300/0 -465> 2019-11-07 16:24:14.776 7f811f851dc0 5 monclient: authenticate success, global_id 491342 -464> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: handle_config config(1 keys) v1 -463> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: handle_monmap mon_map magic: 0 v1 -462> 2019-11-07 16:24:14.776 7f81163cc700 4 set_mon_vals no callback set -461> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: got monmap 8 from mon.qn-cnfslhc (according to old e8) -460> 2019-11-07 16:24:14.776 7f8117bcf700 10 monclient: dump: epoch 8 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-10-08 17:19:02.427513 created 2019-10-03 15:53:10.107437 min_mon_release 14 (nautilus) 0: [v2:131.154.128.131:3300/0,v1:131.154.128.131:6789/0] mon.ds-507 1: [v2:131.154.128.145:3300/0,v1:131.154.128.145:6789/0] mon.ds-001 2: [v2:131.154.130.69:3300/0,v1:131.154.130.69:6789/0] mon.qn-cnfslhc 3: [v2:131.154.130.77:3300/0,v1:131.154.130.77:6789/0] mon.ds-303 4: [v2:131.154.130.166:3300/0,v1:131.154.130.166:6789/0] mon.cs-001 -459> 2019-11-07 16:24:14.776 7f81163cc700 10 set_mon_vals debug_auth = 20/20 -458> 2019-11-07 16:24:14.776 7f811f851dc0 10 monclient: get_monmap_and_config success -457> 2019-11-07 16:24:14.776 7f811f851dc0 10 monclient: shutdown -456> 2019-11-07 16:24:14.779 7f811f851dc0 0 set uid:gid to 167:167 (ceph:ceph) -455> 2019-11-07 16:24:14.779 7f811f851dc0 0 ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable), process ceph-osd, pid 2350606 -454> 2019-11-07 16:24:14.779 7f811f851dc0 0 pidfile_write: ignore empty --pid-file -453> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) init /var/run/ceph/ceph-osd.20.asok -452> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) bind_and_listen /var/run/ceph/ceph-osd.20.asok -451> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) register_command 0 hook 0x56038a8e1648 -450> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) register_command version hook 0x56038a8e1648 -449> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) register_command git_version hook 0x56038a8e1648 -448> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) register_command help hook 0x56038a8e22d0 -447> 2019-11-07 16:24:14.814 7f811f851dc0 5 asok(0x56038a988000) register_command get_command_descriptions hook 0x56038a8e2320 -446> 2019-11-07 16:24:14.814 7f8116bcd700 5 asok(0x56038a988000) entry start -445> 2019-11-07 16:24:14.815 7f811f851dc0 5 object store type is bluestore -444> 2019-11-07 16:24:14.816 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -443> 2019-11-07 16:24:14.816 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -442> 2019-11-07 16:24:14.816 7f811f851dc0 0 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -441> 2019-11-07 16:24:14.817 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -440> 2019-11-07 16:24:14.817 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -439> 2019-11-07 16:24:14.817 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel -438> 2019-11-07 16:24:14.817 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db -437> 2019-11-07 16:24:14.817 7f811f851dc0 0 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument -436> 2019-11-07 16:24:14.818 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -435> 2019-11-07 16:24:14.818 7f811f851dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB -434> 2019-11-07 16:24:14.818 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -433> 2019-11-07 16:24:14.818 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -432> 2019-11-07 16:24:14.819 7f811f851dc0 0 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -431> 2019-11-07 16:24:14.819 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -430> 2019-11-07 16:24:14.819 7f811f851dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB -429> 2019-11-07 16:24:14.819 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel -428> 2019-11-07 16:24:14.819 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal -427> 2019-11-07 16:24:14.819 7f811f851dc0 0 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument -426> 2019-11-07 16:24:14.820 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -425> 2019-11-07 16:24:14.820 7f811f851dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB -424> 2019-11-07 16:24:14.821 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.wal) close -423> 2019-11-07 16:24:15.078 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block.db) close -422> 2019-11-07 16:24:15.323 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block) close -421> 2019-11-07 16:24:15.577 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) close -420> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx -419> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_method_list cephx -> [2] -418> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx -417> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_method_list cephx -> [2] -416> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding auth protocol: cephx -415> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_method_list cephx -> [2] -414> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -413> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -412> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list secure crc -> [2,1] -411> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -410> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -409> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list secure crc -> [2,1] -408> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -407> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -406> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list secure crc -> [2,1] -405> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -404> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -403> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list crc secure -> [1,2] -402> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -401> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -400> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list crc secure -> [1,2] -399> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: crc -398> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a964a38) adding con mode: secure -397> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a964a38) _parse_mode_list crc secure -> [1,2] -396> 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a964a38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -395> 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a964a38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -394> 2019-11-07 16:24:15.824 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -393> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding auth protocol: cephx -392> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_method_list cephx -> [2] -391> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding auth protocol: cephx -390> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_method_list cephx -> [2] -389> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding auth protocol: cephx -388> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_method_list cephx -> [2] -387> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure -386> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc -385> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list secure crc -> [2,1] -384> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure -383> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc -382> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list secure crc -> [2,1] -381> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure -380> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc -379> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list secure crc -> [2,1] -378> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc -377> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure -376> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list crc secure -> [1,2] -375> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc -374> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure -373> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list crc secure -> [1,2] -372> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: crc -371> 2019-11-07 16:24:15.824 7f811f851dc0 5 AuthRegistry(0x56038a965338) adding con mode: secure -370> 2019-11-07 16:24:15.824 7f811f851dc0 20 AuthRegistry(0x56038a965338) _parse_mode_list crc secure -> [1,2] -369> 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a965338) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -368> 2019-11-07 16:24:15.824 7f811f851dc0 10 AuthRegistry(0x56038a965338) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -367> 2019-11-07 16:24:15.824 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -366> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding auth protocol: cephx -365> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_method_list cephx -> [2] -364> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding auth protocol: cephx -363> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_method_list cephx -> [2] -362> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding auth protocol: cephx -361> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_method_list cephx -> [2] -360> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure -359> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc -358> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list secure crc -> [2,1] -357> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure -356> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc -355> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list secure crc -> [2,1] -354> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure -353> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc -352> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list secure crc -> [2,1] -351> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc -350> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure -349> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list crc secure -> [1,2] -348> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc -347> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure -346> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list crc secure -> [1,2] -345> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: crc -344> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a965c38) adding con mode: secure -343> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a965c38) _parse_mode_list crc secure -> [1,2] -342> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a965c38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -341> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a965c38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -340> 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -339> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding auth protocol: cephx -338> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_method_list cephx -> [2] -337> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding auth protocol: cephx -336> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_method_list cephx -> [2] -335> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding auth protocol: cephx -334> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_method_list cephx -> [2] -333> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure -332> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc -331> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list secure crc -> [2,1] -330> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure -329> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc -328> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list secure crc -> [2,1] -327> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure -326> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc -325> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list secure crc -> [2,1] -324> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc -323> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure -322> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list crc secure -> [1,2] -321> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc -320> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure -319> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list crc secure -> [1,2] -318> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: crc -317> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966538) adding con mode: secure -316> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966538) _parse_mode_list crc secure -> [1,2] -315> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966538) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -314> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966538) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -313> 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -312> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding auth protocol: cephx -311> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_method_list cephx -> [2] -310> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding auth protocol: cephx -309> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_method_list cephx -> [2] -308> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding auth protocol: cephx -307> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_method_list cephx -> [2] -306> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure -305> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc -304> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list secure crc -> [2,1] -303> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure -302> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc -301> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list secure crc -> [2,1] -300> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure -299> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc -298> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list secure crc -> [2,1] -297> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc -296> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure -295> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list crc secure -> [1,2] -294> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc -293> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure -292> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list crc secure -> [1,2] -291> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: crc -290> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a966e38) adding con mode: secure -289> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a966e38) _parse_mode_list crc secure -> [1,2] -288> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966e38) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -287> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a966e38) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -286> 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -285> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding auth protocol: cephx -284> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_method_list cephx -> [2] -283> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding auth protocol: cephx -282> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_method_list cephx -> [2] -281> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding auth protocol: cephx -280> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_method_list cephx -> [2] -279> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure -278> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc -277> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list secure crc -> [2,1] -276> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure -275> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc -274> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list secure crc -> [2,1] -273> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure -272> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc -271> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list secure crc -> [2,1] -270> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc -269> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure -268> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list crc secure -> [1,2] -267> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc -266> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure -265> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list crc secure -> [1,2] -264> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: crc -263> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038a967738) adding con mode: secure -262> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038a967738) _parse_mode_list crc secure -> [1,2] -261> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a967738) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -260> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038a967738) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -259> 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -258> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding auth protocol: cephx -257> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_method_list cephx -> [2] -256> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding auth protocol: cephx -255> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_method_list cephx -> [2] -254> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding auth protocol: cephx -253> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_method_list cephx -> [2] -252> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure -251> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc -250> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list secure crc -> [2,1] -249> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure -248> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc -247> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list secure crc -> [2,1] -246> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure -245> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc -244> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list secure crc -> [2,1] -243> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc -242> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure -241> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list crc secure -> [1,2] -240> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc -239> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure -238> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list crc secure -> [1,2] -237> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: crc -236> 2019-11-07 16:24:15.825 7f811f851dc0 5 AuthRegistry(0x56038b676138) adding con mode: secure -235> 2019-11-07 16:24:15.825 7f811f851dc0 20 AuthRegistry(0x56038b676138) _parse_mode_list crc secure -> [1,2] -234> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038b676138) _refresh_config cluster_methods [2] service_methods [2] client_methods [2] -233> 2019-11-07 16:24:15.825 7f811f851dc0 10 AuthRegistry(0x56038b676138) _refresh_config mon_cluster_modes [2,1] mon_service_modes [2,1] mon_client_modes [2,1]; cluster_modes [1,2] service_modes [1,2] client_modes [1,2] -232> 2019-11-07 16:24:15.825 7f811f851dc0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-20/keyring -231> 2019-11-07 16:24:15.825 7f811f851dc0 0 starting osd.20 osd_data /var/lib/ceph/osd/ceph-20 /var/lib/ceph/osd/ceph-20/journal -230> 2019-11-07 16:24:15.825 7f811f851dc0 -1 Falling back to public interface -229> 2019-11-07 16:24:15.858 7f811f851dc0 10 monclient: build_initial_monmap -228> 2019-11-07 16:24:15.858 7f811f851dc0 10 monclient: monmap: epoch 0 fsid 34137c93-4c2d-42f0-85a9-c6b3717e80f8 last_changed 2019-11-07 16:24:15.859855 created 2019-11-07 16:24:15.859855 min_mon_release 0 (unknown) 0: v1:131.154.128.145:6789/0 mon.noname-b-legacy 1: v1:131.154.130.69:6789/0 mon.noname-a-legacy 2: v2:131.154.128.145:3300/0 mon.noname-b 3: v2:131.154.130.69:3300/0 mon.noname-a -227> 2019-11-07 16:24:15.869 7f811f851dc0 0 load: jerasure load: lrc load: isa -226> 2019-11-07 16:24:15.869 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -225> 2019-11-07 16:24:15.869 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -224> 2019-11-07 16:24:15.869 7f811f851dc0 0 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -223> 2019-11-07 16:24:15.869 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -222> 2019-11-07 16:24:15.870 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -221> 2019-11-07 16:24:15.870 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) close -220> 2019-11-07 16:24:16.131 7f811f851dc0 5 asok(0x56038a988000) register_command objecter_requests hook 0x56038a8e2a60 -219> 2019-11-07 16:24:16.132 7f811f851dc0 2 osd.20 0 init /var/lib/ceph/osd/ceph-20 (looks like hdd) -218> 2019-11-07 16:24:16.132 7f811f851dc0 2 osd.20 0 journal /var/lib/ceph/osd/ceph-20/journal -217> 2019-11-07 16:24:16.132 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _mount path /var/lib/ceph/osd/ceph-20 -216> 2019-11-07 16:24:16.132 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -215> 2019-11-07 16:24:16.132 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -214> 2019-11-07 16:24:16.132 7f811f851dc0 0 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -213> 2019-11-07 16:24:16.133 7f811f851dc0 1 bdev(0x56038b532000 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -212> 2019-11-07 16:24:16.133 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 -211> 2019-11-07 16:24:16.133 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.db type kernel -210> 2019-11-07 16:24:16.133 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.db) open path /var/lib/ceph/osd/ceph-20/block.db -209> 2019-11-07 16:24:16.133 7f811f851dc0 0 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.db) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.db failed: (22) Invalid argument -208> 2019-11-07 16:24:16.134 7f811f851dc0 1 bdev(0x56038b532e00 /var/lib/ceph/osd/ceph-20/block.db) open size 10737418240 (0x280000000, 10 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -207> 2019-11-07 16:24:16.134 7f811f851dc0 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-20/block.db size 10 GiB -206> 2019-11-07 16:24:16.135 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block type kernel -205> 2019-11-07 16:24:16.135 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block) open path /var/lib/ceph/osd/ceph-20/block -204> 2019-11-07 16:24:16.135 7f811f851dc0 0 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block failed: (22) Invalid argument -203> 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev(0x56038b532700 /var/lib/ceph/osd/ceph-20/block) open size 8001524072448 (0x74700000000, 7.3 TiB) block_size 4096 (4 KiB) rotational discard not supported -202> 2019-11-07 16:24:16.136 7f811f851dc0 1 bluefs add_block_device bdev 2 path /var/lib/ceph/osd/ceph-20/block size 7.3 TiB -201> 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev create path /var/lib/ceph/osd/ceph-20/block.wal type kernel -200> 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block.wal) open path /var/lib/ceph/osd/ceph-20/block.wal -199> 2019-11-07 16:24:16.136 7f811f851dc0 0 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block.wal) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-20/block.wal failed: (22) Invalid argument -198> 2019-11-07 16:24:16.136 7f811f851dc0 1 bdev(0x56038b532a80 /var/lib/ceph/osd/ceph-20/block.wal) open size 13958643712 (0x340000000, 13 GiB) block_size 4096 (4 KiB) non-rotational discard not supported -197> 2019-11-07 16:24:16.136 7f811f851dc0 1 bluefs add_block_device bdev 0 path /var/lib/ceph/osd/ceph-20/block.wal size 13 GiB -196> 2019-11-07 16:24:16.137 7f811f851dc0 1 bluefs mount -195> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compaction_readahead_size = 2097152 -194> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compression = kNoCompression -193> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_background_compactions = 2 -192> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_write_buffer_number = 4 -191> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 -190> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option recycle_log_file_num = 4 -189> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option writable_file_max_buffer_size = 0 -188> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option write_buffer_size = 268435456 -187> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compaction_readahead_size = 2097152 -186> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option compression = kNoCompression -185> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_background_compactions = 2 -184> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option max_write_buffer_number = 4 -183> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option min_write_buffer_number_to_merge = 1 -182> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option recycle_log_file_num = 4 -181> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option writable_file_max_buffer_size = 0 -180> 2019-11-07 16:24:16.170 7f811f851dc0 0 set rocksdb option write_buffer_size = 268435456 -179> 2019-11-07 16:24:16.170 7f811f851dc0 1 rocksdb: do_open column families: [default] -178> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: RocksDB version: 6.1.2 -177> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ -176> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Compile date Sep 13 2019 -175> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: DB SUMMARY -174> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: CURRENT file: CURRENT -173> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: IDENTITY file: IDENTITY -172> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: MANIFEST file: MANIFEST-003676 size: 463 Bytes -171> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: SST files in db dir, Total Num: 5, files: 003598.sst 003600.sst 003602.sst 003604.sst 003606.sst -170> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: -169> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Write Ahead Log file in db.wal: 003677.log size: 0 ; -168> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.error_if_exists: 0 -167> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.create_if_missing: 0 -166> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.paranoid_checks: 1 -165> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.env: 0x56038b624a60 -164> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.info_log: 0x56038b625200 -163> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_file_opening_threads: 16 -162> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.statistics: (nil) -161> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_fsync: 0 -160> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_log_file_size: 0 -159> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_manifest_file_size: 1073741824 -158> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.log_file_time_to_roll: 0 -157> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.keep_log_file_num: 1000 -156> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.recycle_log_file_num: 4 -155> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_fallocate: 1 -154> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_mmap_reads: 0 -153> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_mmap_writes: 0 -152> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_direct_reads: 0 -151> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 -150> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.create_missing_column_families: 0 -149> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.db_log_dir: -148> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_dir: db.wal -147> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.table_cache_numshardbits: 6 -146> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_subcompactions: 1 -145> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_background_flushes: -1 -144> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.WAL_ttl_seconds: 0 -143> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.WAL_size_limit_MB: 0 -142> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.manifest_preallocation_size: 4194304 -141> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.is_fd_close_on_exec: 1 -140> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.advise_random_on_open: 1 -139> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.db_write_buffer_size: 0 -138> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_buffer_manager: 0x56038b940360 -137> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.access_hint_on_compaction_start: 1 -136> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1 -135> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 -134> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.use_adaptive_mutex: 0 -133> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.rate_limiter: (nil) -132> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 -131> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_recovery_mode: 0 -130> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.enable_thread_tracking: 0 -129> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.enable_pipelined_write: 0 -128> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 -127> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 -126> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_thread_max_yield_usec: 100 -125> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 -124> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.row_cache: None -123> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_filter: None -122> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.avoid_flush_during_recovery: 0 -121> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.allow_ingest_behind: 0 -120> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.preserve_deletes: 0 -119> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.two_write_queues: 0 -118> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.manual_wal_flush: 0 -117> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.atomic_flush: 0 -116> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 -115> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_background_jobs: 2 -114> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_background_compactions: 2 -113> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 -112> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.writable_file_max_buffer_size: 0 -111> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.delayed_write_rate : 16777216 -110> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_total_wal_size: 0 -109> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 -108> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.stats_dump_period_sec: 600 -107> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.stats_persist_period_sec: 600 -106> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.stats_history_buffer_size: 1048576 -105> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_open_files: -1 -104> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bytes_per_sync: 0 -103> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.wal_bytes_per_sync: 0 -102> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_readahead_size: 2097152 -101> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Compression algorithms supported: -100> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kZSTDNotFinalCompression supported: 0 -99> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kZSTD supported: 0 -98> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kXpressCompression supported: 0 -97> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kLZ4HCCompression supported: 1 -96> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kLZ4Compression supported: 1 -95> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kBZip2Compression supported: 0 -94> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kZlibCompression supported: 1 -93> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: kSnappyCompression supported: 1 -92> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Fast CRC32 supported: Supported on x86 -91> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: [db/db_impl_readonly.cc:22] Opening the db in read only mode -90> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-003676 -89> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: -88> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator -87> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor -86> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_filter: None -85> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_filter_factory: None -84> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_factory: SkipListFactory -83> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.table_factory: BlockBasedTable -82> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56038a8e0de0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 1 pin_l0_filter_and_index_blocks_in_cache: 1 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 data_block_hash_table_util_ratio: 0.750000 hash_index_allow_collision: 1 checksum: 1 no_block_cache: 0 block_cache: 0x56038a8f5210 block_cache_name: BinnedLRUCache block_cache_options: capacity : 429496729 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: rocksdb.BuiltinBloomFilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 2 enable_index_compression: 1 block_align: 0 -81> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.write_buffer_size: 268435456 -80> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_write_buffer_number: 4 -79> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression: NoCompression -78> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression: Disabled -77> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.prefix_extractor: nullptr -76> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr -75> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.num_levels: 7 -74> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 -73> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 -72> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 -71> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 -70> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 -69> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 -68> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 -67> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bottommost_compression_opts.enabled: false -66> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.window_bits: -14 -65> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.level: 32767 -64> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.strategy: 0 -63> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 -62> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 -61> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compression_opts.enabled: false -60> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 -59> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 -58> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level0_stop_writes_trigger: 36 -57> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.target_file_size_base: 67108864 -56> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.target_file_size_multiplier: 1 -55> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 -54> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 -53> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 -52> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 -51> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 -50> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 -49> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 -48> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 -47> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 -46> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 -45> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 -44> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_compaction_bytes: 1677721600 -43> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.arena_block_size: 33554432 -42> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 -41> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 -40> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 -39> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.disable_auto_compactions: 0 -38> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel -37> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio -36> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 -35> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 -34> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 -33> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 -32> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 -31> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize -30> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 -29> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 -28> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.table_properties_collectors: -27> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.inplace_update_support: 0 -26> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.inplace_update_num_locks: 10000 -25> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 -24> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_whole_key_filtering: 0 -23> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.memtable_huge_page_size: 0 -22> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.bloom_locality: 0 -21> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.max_successive_merges: 0 -20> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.optimize_filters_for_hits: 0 -19> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.paranoid_file_checks: 0 -18> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.force_consistency_checks: 0 -17> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.report_bg_io_stats: 0 -16> 2019-11-07 16:24:16.171 7f811f851dc0 4 rocksdb: Options.ttl: 0 -15> 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-003676 succeeded,manifest_file_number is 3676, next_file_number is 3678, last_sequence is 293760899, log_number is 3675,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 -14> 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 3675 -13> 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140256220733, "job": 1, "event": "recovery_started", "log_files": [3677]} -12> 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3677 mode 0 -11> 2019-11-07 16:24:16.219 7f811f851dc0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1573140256220762, "job": 1, "event": "recovery_finished"} -10> 2019-11-07 16:24:16.219 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2 -9> 2019-11-07 16:24:16.242 7f811f851dc0 1 get compressor snappy = 0 -8> 2019-11-07 16:24:16.242 7f811f851dc0 1 load compressor snappy -7> 2019-11-07 16:24:16.243 7f811f851dc0 1 add compressor snappy 0x56038b940960 -6> 2019-11-07 16:24:16.243 7f811f851dc0 1 get compressor snappy = 0x56038b940960 -5> 2019-11-07 16:24:16.243 7f811f851dc0 1 load: compressor snappy loaded and registered -4> 2019-11-07 16:24:16.243 7f811f851dc0 1 get compressor snappy = 0x56038b940960 -3> 2019-11-07 16:24:16.243 7f811f851dc0 1 freelist init -2> 2019-11-07 16:24:17.621 7f811f851dc0 1 bluestore(/var/lib/ceph/osd/ceph-20) _open_alloc opening allocation metadata -1> 2019-11-07 16:24:17.684 7f811f851dc0 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: In function 'void AllocatorLevel02::_mark_l2_free(int64_t, int64_t) [with L1 = AllocatorLevel01Loose; int64_t = long int]' thread 7f811f851dc0 time 2019-11-07 16:24:17.681601 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.4/rpm/el7/BUILD/ceph-14.2.4/src/os/bluestore/fastbmap_allocator_impl.h: 584: FAILED ceph_assert((int64_t)l2.size() >= (l2_pos_end / d)) ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x56037f236a24] 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x56037f236bf2] 3: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x56037f867870] 4: (BlueStore::_open_alloc()+0x258) [0x56037f70dab8] 5: (BlueStore::_open_db_and_around(bool)+0x146) [0x56037f72f306] 6: (BlueStore::_mount(bool, bool)+0x6a4) [0x56037f76dc24] 7: (OSD::init()+0x3aa) [0x56037f2e0efa] 8: (main()+0x14fa) [0x56037f23b1da] 9: (__libc_start_main()+0xf5) [0x7f811b4c3c05] 10: (()+0x4b2695) [0x56037f270695] 0> 2019-11-07 16:24:17.689 7f811f851dc0 -1 *** Caught signal (Aborted) ** in thread 7f811f851dc0 thread_name:ceph-osd ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable) 1: (()+0xf5e0) [0x7f811c4c45e0] 2: (gsignal()+0x37) [0x7f811b4d71f7] 3: (abort()+0x148) [0x7f811b4d88e8] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x56037f236a73] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x56037f236bf2] 6: (BitmapAllocator::init_add_free(unsigned long, unsigned long)+0x740) [0x56037f867870] 7: (BlueStore::_open_alloc()+0x258) [0x56037f70dab8] 8: (BlueStore::_open_db_and_around(bool)+0x146) [0x56037f72f306] 9: (BlueStore::_mount(bool, bool)+0x6a4) [0x56037f76dc24] 10: (OSD::init()+0x3aa) [0x56037f2e0efa] 11: (main()+0x14fa) [0x56037f23b1da] 12: (__libc_start_main()+0xf5) [0x7f811b4c3c05] 13: (()+0x4b2695) [0x56037f270695] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 0/ 0 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 20/20 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 rgw_sync 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.20.log --- end dump of recent events --- [root@cs-001 ~]#