May 04 15:40:05 02.ceph-kubernetes.dev.lax1.REDACTED.net systemd[1]: Started Ceph osd.2 for 87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef. May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/ceph --cluster ceph --name client.osd-lockbox.3c48b9f9-ee36-416f-a653-8d018cbf8875 --keyring /var/lib/ceph/osd/ceph-2/lockbox.keyring config-key get dm-crypt/osd/3c48b9f9-ee36-416f-a653-8d018cbf8875/luks May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/sbin/cryptsetup --key-file - --allow-discards luksOpen /dev/ceph-add3a9f2-0da2-4675-ab9c-fe5aff30b6f3/osd-block-3c48b9f9-ee36-416f-a653-8d018cbf8875 gTCQdK-HJgL-zLfg-EkTI-1BHu-QdPu-afsGdI May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: stderr: Device gTCQdK-HJgL-zLfg-EkTI-1BHu-QdPu-afsGdI already exists. May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/mapper/gTCQdK-HJgL-zLfg-EkTI-1BHu-QdPu-afsGdI --path /var/lib/ceph/osd/ceph-2 --no-mon-config May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/ln -snf /dev/mapper/gTCQdK-HJgL-zLfg-EkTI-1BHu-QdPu-afsGdI /var/lib/ceph/osd/ceph-2/block May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32525]: --> ceph-volume lvm activate successful for osd ID: 2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: *** Caught signal (Segmentation fault) ** May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: in thread 7f4e61e54080 thread_name:ceph-osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug 2021-05-04T15:40:07.910+0000 7f4e61e54080 0 set uid:gid to 167:167 (ceph:ceph) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug 2021-05-04T15:40:07.910+0000 7f4e61e54080 0 ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable), process ceph-osd, pid 6 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: /lib64/libpthread.so.0(+0x12b20) [0x7f4e5fbbbb20] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: /lib64/libc.so.6(+0x9a3da) [0x7f4e5e8863da] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 3: (SubProcess::add_cmd_arg(char const*)+0x4c) [0x56504e693b2c] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4: (SubProcess::add_cmd_args(char const*, ...)+0x75) [0x56504e693cc5] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 5: (ceph::crush::CrushLocation::update_from_hook()+0x2d4) [0x56504e883304] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 6: (ceph::crush::CrushLocation::init_on_startup()+0x3f5) [0x56504e884455] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 7: (global_init(std::map, std::allocator >, std::__cxx11::basic_string, std::allocator >, std::less, std::allocator > >, std::allocator, std::allocator > const, std::__cxx11::basic_string, std::allocator > > > > const*, std::vector >&, unsigned int, code_environment_t, int, bool)+0xcd1) [0x56504e5305b1] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 8: main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 9: __libc_start_main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 10: _start() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug 2021-05-04T15:40:07.914+0000 7f4e61e54080 -1 *** Caught signal (Segmentation fault) ** May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: in thread 7f4e61e54080 thread_name:ceph-osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: /lib64/libpthread.so.0(+0x12b20) [0x7f4e5fbbbb20] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: /lib64/libc.so.6(+0x9a3da) [0x7f4e5e8863da] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 3: (SubProcess::add_cmd_arg(char const*)+0x4c) [0x56504e693b2c] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4: (SubProcess::add_cmd_args(char const*, ...)+0x75) [0x56504e693cc5] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 5: (ceph::crush::CrushLocation::update_from_hook()+0x2d4) [0x56504e883304] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 6: (ceph::crush::CrushLocation::init_on_startup()+0x3f5) [0x56504e884455] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 7: (global_init(std::map, std::allocator >, std::__cxx11::basic_string, std::allocator >, std::less, std::allocator > >, std::allocator, std::allocator > const, std::__cxx11::basic_string, std::allocator > > > > const*, std::vector >&, unsigned int, code_environment_t, int, bool)+0xcd1) [0x56504e5305b1] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 8: main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 9: __libc_start_main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 10: _start() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- begin dump of recent events --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -104> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command assert hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -103> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command abort hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -102> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command leak_some_memory hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -101> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perfcounters_dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -100> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command 1 hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -99> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -98> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perfcounters_schema hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -97> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf histogram dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -96> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command 2 hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -95> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf schema hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -94> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf histogram schema hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -93> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf reset hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -92> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config show hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -91> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config help hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -90> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config set hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -89> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config unset hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -88> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config get hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -87> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config diff hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -86> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config diff get hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -85> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command injectargs hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -84> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command log flush hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -83> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command log dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -82> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command log reopen hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -81> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command dump_mempools hook 0x565058206328 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -80> 2021-05-04T15:40:07.906+0000 7f4e61e54080 10 monclient: get_monmap_and_config May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -79> 2021-05-04T15:40:07.906+0000 7f4e61e54080 10 monclient: build_initial_monmap May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -78> 2021-05-04T15:40:07.906+0000 7f4e61e54080 10 monclient: monmap: May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: epoch 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: fsid 87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: last_changed 2021-05-04T15:40:07.910610+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: created 2021-05-04T15:40:07.910610+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: min_mon_release 0 (unknown) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: election_strategy: 1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0: [v2:10.0.114.9:3300/0,v1:10.0.114.9:6789/0] mon.noname-a May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: [v2:10.0.114.16:3300/0,v1:10.0.114.16:6789/0] mon.noname-b May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: [v2:10.0.114.17:3300/0,v1:10.0.114.17:6789/0] mon.noname-c May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -77> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -76> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -75> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -74> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: none May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -73> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -72> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -71> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -70> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -69> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -68> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -67> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -66> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -65> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -64> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -63> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -62> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -61> 2021-05-04T15:40:07.906+0000 7f4e61e54080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -60> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: init May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -59> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -58> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -57> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -56> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: none May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -55> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -54> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -53> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -52> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -51> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -50> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -49> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -48> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -47> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -46> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -45> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -44> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -43> 2021-05-04T15:40:07.910+0000 7f4e61e54080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -42> 2021-05-04T15:40:07.910+0000 7f4e61e54080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -41> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: _reopen_session rank -1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -40> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: _add_conns ranks=[0,1,2] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -39> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): picked mon.noname-a con 0x565058214c00 addr [v2:10.0.114.9:3300/0,v1:10.0.114.9:6789/0] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -38> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): picked mon.noname-b con 0x565058215000 addr [v2:10.0.114.16:3300/0,v1:10.0.114.16:6789/0] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -37> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): picked mon.noname-c con 0x565058215400 addr [v2:10.0.114.17:3300/0,v1:10.0.114.17:6789/0] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -36> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): start opening mon connection May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -35> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): start opening mon connection May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -34> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): start opening mon connection May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -33> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): _renew_subs May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -32> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): authenticate will time out at 2021-05-04T15:45:07.912128+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -31> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): get_auth_request con 0x565058215000 auth_method 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -30> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -29> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): _init_auth method 2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -28> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): _init_auth creating new auth May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -27> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_reply_more payload 9 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -26> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_reply_more payload_len 9 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -25> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -24> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_done global_id 1316491 payload 274 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -23> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient: _finish_hunting 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -22> 2021-05-04T15:40:07.910+0000 7f4e5abca700 1 monclient: found mon.noname-b May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -21> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient: _send_mon_message to mon.noname-b at v2:10.0.114.16:3300/0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -20> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: handle_monmap mon_map magic: 0 v1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -19> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: got monmap 13 from mon.noname-b (according to old e13) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -18> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: dump: May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: epoch 13 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: fsid 87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: last_changed 2021-04-29T15:09:01.891190+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: created 2020-12-14T18:42:30.246595+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: min_mon_release 16 (pacific) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: election_strategy: 1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0: [v2:10.0.114.17:3300/0,v1:10.0.114.17:6789/0] mon.03 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: [v2:10.0.114.16:3300/0,v1:10.0.114.16:6789/0] mon.02 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: [v2:10.0.114.9:3300/0,v1:10.0.114.9:6789/0] mon.01 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -17> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: _finish_auth 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -16> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-05-04T15:39:37.913786+0000) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -15> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: _send_mon_message to mon.02 at v2:10.0.114.16:3300/0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -14> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 monclient: authenticate success, global_id 1316491 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -13> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: handle_config config(7 keys) v1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -12> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: get_monmap_and_config success May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -11> 2021-05-04T15:40:07.910+0000 7f4e61e54080 4 set_mon_vals no callback set May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -10> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals cluster_network = 10.0.0.0/8 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -9> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals container_image = docker.artifactory.prod.adnxs.net/ceph/ceph@sha256:e5a456db2779401b485345a4eadc595b611b48faf7304bf6d246fd4707e896c7 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -8> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals crush_location_hook = /var/log/ceph/crush_location May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -7> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals osd_memory_target = 4294967296 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -6> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals osd_pool_default_crush_rule = 1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -5> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals osd_pool_default_pg_autoscale_mode = warn May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -4> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals public_network = 10.0.0.0/8 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -3> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: shutdown May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -2> 2021-05-04T15:40:07.910+0000 7f4e61e54080 0 set uid:gid to 167:167 (ceph:ceph) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -1> 2021-05-04T15:40:07.910+0000 7f4e61e54080 0 ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable), process ceph-osd, pid 6 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug 0> 2021-05-04T15:40:07.914+0000 7f4e61e54080 -1 *** Caught signal (Segmentation fault) ** May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: in thread 7f4e61e54080 thread_name:ceph-osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: /lib64/libpthread.so.0(+0x12b20) [0x7f4e5fbbbb20] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: /lib64/libc.so.6(+0x9a3da) [0x7f4e5e8863da] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 3: (SubProcess::add_cmd_arg(char const*)+0x4c) [0x56504e693b2c] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4: (SubProcess::add_cmd_args(char const*, ...)+0x75) [0x56504e693cc5] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 5: (ceph::crush::CrushLocation::update_from_hook()+0x2d4) [0x56504e883304] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 6: (ceph::crush::CrushLocation::init_on_startup()+0x3f5) [0x56504e884455] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 7: (global_init(std::map, std::allocator >, std::__cxx11::basic_string, std::allocator >, std::less, std::allocator > >, std::allocator, std::allocator > const, std::__cxx11::basic_string, std::allocator > > > > const*, std::vector >&, unsigned int, code_environment_t, int, bool)+0xcd1) [0x56504e5305b1] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 8: main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 9: __libc_start_main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 10: _start() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- logging levels --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 none May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 lockdep May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 context May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 crush May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_balancer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_locker May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_log May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_log_expire May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_migrator May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 buffer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 timer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 filer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 striper May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 objecter May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rados May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd_mirror May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd_replay May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd_pwl May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 journaler May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 objectcacher May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 immutable_obj_cache May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 client May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 optracker May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 objclass May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 3 filestore May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 3 journal May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 0 ms May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mon May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/10 monc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 paxos May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 tp May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 auth May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 crypto May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 finisher May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 reserver May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 heartbeatmap May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 perfcounter May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 rgw May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 rgw_sync May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/10 civetweb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 javaclient May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 asok May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 throttle May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 0 refs May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 compressor May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 bluestore May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 bluefs May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 3 bdev May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 kstore May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4/ 5 rocksdb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4/ 5 leveldb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4/ 5 memdb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 fuse May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mgr May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mgrc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 dpdk May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 eventtrace May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 prioritycache May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 test May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 cephfs_mirror May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 cephsqlite May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: -2/-2 (syslog threshold) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 99/99 (stderr threshold) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- pthread ID / name mapping for recent threads --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 139974498096896 / May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 139974506489600 / msgr-worker-2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 139974626590848 / ceph-osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: max_recent 10000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: max_new 10000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: log_file May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- end dump of recent events --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- begin dump of recent events --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -104> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command assert hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -103> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command abort hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -102> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command leak_some_memory hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -101> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perfcounters_dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -100> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command 1 hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -99> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -98> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perfcounters_schema hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -97> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf histogram dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -96> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command 2 hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -95> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf schema hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -94> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf histogram schema hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -93> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command perf reset hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -92> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config show hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -91> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config help hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -90> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config set hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -89> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config unset hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -88> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config get hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -87> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config diff hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -86> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command config diff get hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -85> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command injectargs hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -84> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command log flush hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -83> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command log dump hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -82> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command log reopen hook 0x565058166540 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -81> 2021-05-04T15:40:07.898+0000 7f4e61e54080 5 asok(0x565058218000) register_command dump_mempools hook 0x565058206328 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -80> 2021-05-04T15:40:07.906+0000 7f4e61e54080 10 monclient: get_monmap_and_config May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -79> 2021-05-04T15:40:07.906+0000 7f4e61e54080 10 monclient: build_initial_monmap May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -78> 2021-05-04T15:40:07.906+0000 7f4e61e54080 10 monclient: monmap: May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: epoch 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: fsid 87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: last_changed 2021-05-04T15:40:07.910610+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: created 2021-05-04T15:40:07.910610+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: min_mon_release 0 (unknown) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: election_strategy: 1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0: [v2:10.0.114.9:3300/0,v1:10.0.114.9:6789/0] mon.noname-a May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: [v2:10.0.114.16:3300/0,v1:10.0.114.16:6789/0] mon.noname-b May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: [v2:10.0.114.17:3300/0,v1:10.0.114.17:6789/0] mon.noname-c May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -77> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -76> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -75> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -74> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding auth protocol: none May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -73> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -72> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -71> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -70> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -69> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -68> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -67> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -66> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -65> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -64> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -63> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -62> 2021-05-04T15:40:07.906+0000 7f4e61e54080 5 AuthRegistry(0x565058dea140) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -61> 2021-05-04T15:40:07.906+0000 7f4e61e54080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -60> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: init May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -59> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -58> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -57> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: cephx May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -56> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding auth protocol: none May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -55> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -54> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -53> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -52> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -51> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -50> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -49> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -48> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -47> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -46> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -45> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: crc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -44> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 AuthRegistry(0x7ffdc3d36430) adding con mode: secure May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -43> 2021-05-04T15:40:07.910+0000 7f4e61e54080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -42> 2021-05-04T15:40:07.910+0000 7f4e61e54080 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -41> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: _reopen_session rank -1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -40> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: _add_conns ranks=[0,1,2] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -39> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): picked mon.noname-a con 0x565058214c00 addr [v2:10.0.114.9:3300/0,v1:10.0.114.9:6789/0] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -38> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): picked mon.noname-b con 0x565058215000 addr [v2:10.0.114.16:3300/0,v1:10.0.114.16:6789/0] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -37> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): picked mon.noname-c con 0x565058215400 addr [v2:10.0.114.17:3300/0,v1:10.0.114.17:6789/0] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -36> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): start opening mon connection May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -35> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): start opening mon connection May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -34> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): start opening mon connection May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -33> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): _renew_subs May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -32> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient(hunting): authenticate will time out at 2021-05-04T15:45:07.912128+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -31> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): get_auth_request con 0x565058215000 auth_method 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -30> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -29> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): _init_auth method 2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -28> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): _init_auth creating new auth May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -27> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_reply_more payload 9 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -26> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_reply_more payload_len 9 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -25> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -24> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient(hunting): handle_auth_done global_id 1316491 payload 274 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -23> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient: _finish_hunting 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -22> 2021-05-04T15:40:07.910+0000 7f4e5abca700 1 monclient: found mon.noname-b May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -21> 2021-05-04T15:40:07.910+0000 7f4e5abca700 10 monclient: _send_mon_message to mon.noname-b at v2:10.0.114.16:3300/0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -20> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: handle_monmap mon_map magic: 0 v1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -19> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: got monmap 13 from mon.noname-b (according to old e13) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -18> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: dump: May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: epoch 13 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: fsid 87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: last_changed 2021-04-29T15:09:01.891190+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: created 2020-12-14T18:42:30.246595+0000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: min_mon_release 16 (pacific) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: election_strategy: 1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0: [v2:10.0.114.17:3300/0,v1:10.0.114.17:6789/0] mon.03 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: [v2:10.0.114.16:3300/0,v1:10.0.114.16:6789/0] mon.02 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: [v2:10.0.114.9:3300/0,v1:10.0.114.9:6789/0] mon.01 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -17> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: _finish_auth 0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -16> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2021-05-04T15:39:37.913786+0000) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -15> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: _send_mon_message to mon.02 at v2:10.0.114.16:3300/0 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -14> 2021-05-04T15:40:07.910+0000 7f4e61e54080 5 monclient: authenticate success, global_id 1316491 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -13> 2021-05-04T15:40:07.910+0000 7f4e5a3c9700 10 monclient: handle_config config(7 keys) v1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -12> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: get_monmap_and_config success May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -11> 2021-05-04T15:40:07.910+0000 7f4e61e54080 4 set_mon_vals no callback set May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -10> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals cluster_network = 10.0.0.0/8 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -9> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals container_image = docker.artifactory.prod.adnxs.net/ceph/ceph@sha256:e5a456db2779401b485345a4eadc595b611b48faf7304bf6d246fd4707e896c7 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -8> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals crush_location_hook = /var/log/ceph/crush_location May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -7> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals osd_memory_target = 4294967296 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -6> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals osd_pool_default_crush_rule = 1 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -5> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals osd_pool_default_pg_autoscale_mode = warn May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -4> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 set_mon_vals public_network = 10.0.0.0/8 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -3> 2021-05-04T15:40:07.910+0000 7f4e61e54080 10 monclient: shutdown May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -2> 2021-05-04T15:40:07.910+0000 7f4e61e54080 0 set uid:gid to 167:167 (ceph:ceph) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug -1> 2021-05-04T15:40:07.910+0000 7f4e61e54080 0 ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable), process ceph-osd, pid 6 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: debug 0> 2021-05-04T15:40:07.914+0000 7f4e61e54080 -1 *** Caught signal (Segmentation fault) ** May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: in thread 7f4e61e54080 thread_name:ceph-osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1: /lib64/libpthread.so.0(+0x12b20) [0x7f4e5fbbbb20] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 2: /lib64/libc.so.6(+0x9a3da) [0x7f4e5e8863da] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 3: (SubProcess::add_cmd_arg(char const*)+0x4c) [0x56504e693b2c] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4: (SubProcess::add_cmd_args(char const*, ...)+0x75) [0x56504e693cc5] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 5: (ceph::crush::CrushLocation::update_from_hook()+0x2d4) [0x56504e883304] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 6: (ceph::crush::CrushLocation::init_on_startup()+0x3f5) [0x56504e884455] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 7: (global_init(std::map, std::allocator >, std::__cxx11::basic_string, std::allocator >, std::less, std::allocator > >, std::allocator, std::allocator > const, std::__cxx11::basic_string, std::allocator > > > > const*, std::vector >&, unsigned int, code_environment_t, int, bool)+0xcd1) [0x56504e5305b1] May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 8: main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 9: __libc_start_main() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 10: _start() May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- logging levels --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 none May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 lockdep May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 context May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 crush May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_balancer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_locker May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_log May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_log_expire May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mds_migrator May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 buffer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 timer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 filer May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 striper May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 1 objecter May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rados May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd_mirror May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd_replay May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 rbd_pwl May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 journaler May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 objectcacher May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 immutable_obj_cache May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 client May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 optracker May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 objclass May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 3 filestore May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 3 journal May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 0 ms May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mon May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/10 monc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 paxos May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 tp May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 auth May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 crypto May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 finisher May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 reserver May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 heartbeatmap May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 perfcounter May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 rgw May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 rgw_sync May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/10 civetweb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 javaclient May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 asok May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 1 throttle May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 0 refs May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 compressor May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 bluestore May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 bluefs May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 3 bdev May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 kstore May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4/ 5 rocksdb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4/ 5 leveldb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 4/ 5 memdb May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 fuse May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mgr May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 mgrc May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 dpdk May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 eventtrace May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 1/ 5 prioritycache May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 test May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 cephfs_mirror May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 0/ 5 cephsqlite May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: -2/-2 (syslog threshold) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 99/99 (stderr threshold) May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- pthread ID / name mapping for recent threads --- May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 139974498096896 / May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 139974506489600 / msgr-worker-2 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: 139974626590848 / ceph-osd May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: max_recent 10000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: max_new 10000 May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: log_file /var/lib/ceph/crash/2021-05-04T15:40:07.918919Z_fc38cc93-ef43-4bf3-aed9-0c587d0de619/log May 04 15:40:07 02.ceph-kubernetes.dev.lax1.REDACTED.net bash[32744]: --- end dump of recent events --- May 04 15:40:08 02.ceph-kubernetes.dev.lax1.REDACTED.net systemd[1]: ceph-87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef@osd.2.service: Main process exited, code=exited, status=139/n/a May 04 15:40:09 02.ceph-kubernetes.dev.lax1.REDACTED.net systemd[1]: ceph-87309d4d-e1fc-4be6-9f59-1f77ce2cc5ef@osd.2.service: Failed with result 'exit-code'.