Ceph

After upgrading from Fedora 37 to Fedora 39, I can no longer mount a CEPH volume. rbdmap fails to start:

Nov 14 06:00:21 au1-backup systemd¹: Starting rbdmap.service - Map RBD devices...
Nov 14 06:00:22 au1-backup audit¹²⁴¹: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 pid=1241 comm="rbd" exe="/usr/bin/rbd" sig=11 res=1
Nov 14 06:00:22 au1-backup systemd-coredump¹²⁵⁰: Process 1241 (rbd) of user 0 dumped core.#012#012Module libnss_resolve.so.2 from rpm systemd-254.5-2.fc39.x86_64#012Module libnss_myhostname.so.2 from rpm systemd-254.5-2.fc39.x86_64#012Module liblttng-ust-common.so.1 from rpm lttng-ust-2.13.6-5.fc39.x86_64#012Module liblttng-ust-tracepoint.so.1 from rpm lttng-ust-2.13.6-5.fc39.x86_64#012Module libcrypt.so.2 from rpm libxcrypt-4.4.36-2.fc39.x86_64#012Module libpcre2-8.so.0 from rpm pcre2-10.42-1.fc39.2.x86_64#012Module libbrotlicommon.so.1 from rpm brotli-1.1.0-1.fc39.x86_64#012Module libsasl2.so.3 from rpm cyrus-sasl-2.1.28-11.fc39.x86_64#012Module libevent-2.1.so.7 from rpm libevent-2.1.12-9.fc39.x86_64#012Module libkrb5support.so.0 from rpm krb5-1.21.2-2.fc39.x86_64#012Module libcom_err.so.2 from rpm e2fsprogs-1.47.0-2.fc39.x86_64#012Module libk5crypto.so.3 from rpm krb5-1.21.2-2.fc39.x86_64#012Module libkrb5.so.3 from rpm krb5-1.21.2-2.fc39.x86_64#012Module libunistring.so.5 from rpm libunistring-1.1-5.fc39.x86_64#012Module libselinux.so.1 from rpm libselinux-3.5-5.fc39.x86_64#012Module libbrotlidec.so.1 from rpm brotli-1.1.0-1.fc39.x86_64#012Module libgssapi_krb5.so.2 from rpm krb5-1.21.2-2.fc39.x86_64#012Module libpsl.so.5 from rpm libpsl-0.21.2-4.fc39.x86_64#012Module libssh.so.4 from rpm libssh-0.10.5-2.fc39.x86_64#012Module libidn2.so.0 from rpm libidn2-2.3.4-3.fc39.x86_64#012Module libnghttp2.so.14 from rpm nghttp2-1.55.1-4.fc39.x86_64#012Module libnl-3.so.200 from rpm libnl3-3.8.0-1.fc39.x86_64#012Module libnl-route-3.so.200 from rpm libnl3-3.8.0-1.fc39.x86_64#012Module libjson-c.so.5 from rpm json-c-0.17-1.fc39.x86_64#012Module libargon2.so.1 from rpm argon2-20190702-3.fc39.x86_64#012Module libdevmapper.so.1.02 from rpm lvm2-2.03.22-1.fc39.x86_64#012Module libuuid.so.1 from rpm util-linux-2.39.2-1.fc39.x86_64#012Module libcap.so.2 from rpm libcap-2.48-7.fc39.x86_64#012Module libthrift-0.15.0.so from rpm thrift-0.15.0-3.fc39.x86_64#012Module libcurl.so.4 from rpm curl-8.2.1-3.fc39.x86_64#012Module libz.so.1 from rpm zlib-1.2.13-4.fc39.x86_64#012Module librdmacm.so.1 from rpm rdma-core-46.0-4.fc39.x86_64#012Module libibverbs.so.1 from rpm rdma-core-46.0-4.fc39.x86_64#012Module libcrypto.so.3 from rpm openssl-3.1.1-4.fc39.x86_64#012Module libcryptsetup.so.12 from rpm cryptsetup-2.6.1-3.fc39.x86_64#012Module libssl.so.3 from rpm openssl-3.1.1-4.fc39.x86_64#012Module libkeyutils.so.1 from rpm keyutils-1.6.1-7.fc39.x86_64#012Module libudev.so.1 from rpm systemd-254.5-2.fc39.x86_64#012Module libceph-common.so.2 from rpm ceph-18.2.0-2.fc39.x86_64#012Module libtinfo.so.6 from rpm ncurses-6.4-7.20230520.fc39.x86_64#012Module libncurses.so.6 from rpm ncurses-6.4-7.20230520.fc39.x86_64#012Module libblkid.so.1 from rpm util-linux-2.39.2-1.fc39.x86_64#012Module librados.so.2 from rpm ceph-18.2.0-2.fc39.x86_64#012Module librbd.so.1 from rpm ceph-18.2.0-2.fc39.x86_64#012Module rbd from rpm ceph-18.2.0-2.fc39.x86_64#012Stack trace of thread 1241:#012#0 0x00007fb174eae834 _pthread_kill_implementation (libc.so.6 + 0x90834)#012#1 0x00007fb174e5c8ee raise (libc.so.6 + 0x3e8ee)#012#2 0x0000561c345ca41b _ZL27handle_oneshot_fatal_signali (rbd + 0x25c41b)#012#3 0x00007fb174e5c9a0 __restore_rt (libc.so.6 + 0x3e9a0)#012#4 0x00007fb1750cd904 _ZSt28_Rb_tree_rebalance_for_erasePSt18_Rb_tree_node_baseRS (libstdc++.so.6 + 0xcd904)#012#5 0x00007fb175681799 ZN15CommonSafeTimerISt5mutexE17cancel_all_eventsEv (libceph-common.so.2 + 0x281799)#012#6 0x00007fb175681a71 _ZN15CommonSafeTimerISt5mutexE8shutdownEv (libceph-common.so.2 + 0x281a71)#012#7 0x00007fb1758c6b01 _ZN9MonClient8shutdownEv (libceph-common.so.2 + 0x4c6b01)#012#8 0x00007fb1758c7c07 _ZN9MonClient21get_monmap_and_configEv (libceph-common.so.2 + 0x4c7c07)#012#9 0x00007fb175f3c21e _ZN8librados7v14_2_011RadosClient7connectEv (librados.so.2 + 0xc421e)#012#10 0x0000561c344b1c8c _ZN3rbd5utils10init_radosEPN8librados7v14_2_05RadosE (rbd + 0x143c8c)#012#11 0x0000561c344e8a21 _ZN3rbd6action6kernel11execute_mapERKN5boost15program_options13variables_mapERKSt6vectorINSt7_cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISD_EE (rbd + 0x17aa21)#012#12 0x0000561c344ae76d _ZN3rbd5Shell7executeEiPPKc (rbd + 0x14076d)#012#13 0x0000561c3445024c main (rbd + 0xe224c)#012#14 0x00007fb174e4614a __libc_start_call_main (libc.so.6 + 0x2814a)#012#15 0x00007fb174e4620b __libc_start_main@GLIBC_2.34 (libc.so.6 + 0x2820b)#012#16 0x0000561c344767c5 _start (rbd + 0x1087c5)#012#012Stack trace of thread 1244:#012#0 0x00007fb174f33ac2 epoll_wait (libc.so.6 + 0x115ac2)#012#1 0x00007fb17588501d _ZN11EpollDriver10event_waitERSt6vectorI14FiredFileEventSaIS1_EEP7timeval (libceph-common.so.2 + 0x48501d)#012#2 0x00007fb1758831b4 _ZN11EventCenter14process_eventsEjPNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEE (libceph-common.so.2 + 0x4831b4)#012#3 0x00007fb175883aa9 _ZNSt17_Function_handlerIFvvEZN12NetworkStack10add_threadEP6WorkerEUlvE_E9_M_invokeERKSt9_Any_data (libceph-common.so.2 + 0x483aa9)#012#4 0x00007fb1750e31b3 execute_native_thread_routine (libstdc++.so.6 + 0xe31b3)#012#5 0x00007fb174eac897 start_thread (libc.so.6 + 0x8e897)#012#6 0x00007fb174f336bc __clone3 (libc.so.6 + 0x1156bc)#012#012Stack trace of thread 1245:#012#0 0x00007fb174f33ac2 epoll_wait (libc.so.6 + 0x115ac2)#012#1 0x00007fb17588501d _ZN11EpollDriver10event_waitERSt6vectorI14FiredFileEventSaIS1_EEP7timeval (libceph-common.so.2 + 0x48501d)#012#2 0x00007fb1758831b4 _ZN11EventCenter14process_eventsEjPNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEE (libceph-common.so.2 + 0x4831b4)#012#3 0x00007fb175883aa9 _ZNSt17_Function_handlerIFvvEZN12NetworkStack10add_threadEP6WorkerEUlvE_E9_M_invokeERKSt9_Any_data (libceph-common.so.2 + 0x483aa9)#012#4 0x00007fb1750e31b3 execute_native_thread_routine (libstdc++.so.6 + 0xe31b3)#012#5 0x00007fb174eac897 start_thread (libc.so.6 + 0x8e897)#012#6 0x00007fb174f336bc __clone3 (libc.so.6 + 0x1156bc)#012#012Stack trace of thread 1246:#012#0 0x00007fb174ea9169 __futex_abstimed_wait_common (libc.so.6 + 0x8b169)#012#1 0x00007fb174eabb09 pthread_cond_wait@GLIBC_2.3.2 (libc.so.6 + 0x8db09)#012#2 0x00007fb1750dc180 _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (libstdc++.so.6 + 0xdc180)#012#3 0x00007fb1757aae2c _ZN13DispatchQueue5entryEv (libceph-common.so.2 + 0x3aae2c)#012#4 0x00007fb17583da11 _ZN13DispatchQueue14DispatchThread5entryEv (libceph-common.so.2 + 0x43da11)#012#5 0x00007fb174eac897 start_thread (libc.so.6 + 0x8e897)#012#6 0x00007fb174f336bc __clone3 (libc.so.6 + 0x1156bc)#012#012Stack trace of thread 1247:#012#0 0x00007fb174ea9169 __futex_abstimed_wait_common (libc.so.6 + 0x8b169)#012#1 0x00007fb174eabb09 pthread_cond_wait@GLIBC_2.3.2 (libc.so.6 + 0x8db09)#012#2 0x00007fb1750dc180 _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (libstdc++.so.6 + 0xdc180)#012#3 0x00007fb1757aa7c8 _ZN13DispatchQueue18run_local_deliveryEv (libceph-common.so.2 + 0x3aa7c8)#012#4 0x00007fb17583da31 _ZN13DispatchQueue19LocalDeliveryThread5entryEv (libceph-common.so.2 + 0x43da31)#012#5 0x00007fb174eac897 start_thread (libc.so.6 + 0x8e897)#012#6 0x00007fb174f336bc __clone3 (libc.so.6 + 0x1156bc)#012#012Stack trace of thread 1242:#012#0 0x00007fb174ea9169 __futex_abstimed_wait_common (libc.so.6 + 0x8b169)#012#1 0x00007fb174eabb09 pthread_cond_wait@GLIBC_2.3.2 (libc.so.6 + 0x8db09)#012#2 0x00007fb1750dc180 _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (libstdc++.so.6 + 0xdc180)#012#3 0x00007fb1758afbb6 _ZN4ceph7logging3Log5entryEv (libceph-common.so.2 + 0x4afbb6)#012#4 0x00007fb174eac897 start_thread (libc.so.6 + 0x8e897)#012#5 0x00007fb174f336bc __clone3 (libc.so.6 + 0x1156bc)#012#012Stack trace of thread 1243:#012#0 0x00007fb174f33ac2 epoll_wait (libc.so.6 + 0x115ac2)#012#1 0x00007fb17588501d _ZN11EpollDriver10event_waitERSt6vectorI14FiredFileEventSaIS1_EEP7timeval (libceph-common.so.2 + 0x48501d)#012#2 0x00007fb1758831b4 _ZN11EventCenter14process_eventsEjPNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEE (libceph-common.so.2 + 0x4831b4)#012#3 0x00007fb175883aa9 _ZNSt17_Function_handlerIFvvEZN12NetworkStack10add_thread
Nov 14 06:00:22 au1-backup rbdmap¹²⁶⁷: Failed to map 'rbd/cid-13293-SAU-1B4D6-RS
Nov 14 06:00:22 au1-backup systemd¹: rbdmap.service: Main process exited, code=exited, status=1/FAILURE
Nov 14 06:00:22 au1-backup systemd¹: rbdmap.service: Failed with result 'exit-code'.
Nov 14 06:00:22 au1-backup systemd¹: Failed to start rbdmap.service - Map RBD devices.
Nov 14 06:00:22 au1-backup audit¹: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=rbdmap comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Nov 14 06:00:25 au1-backup abrt-notification¹³³⁷: Process 976 (rbd) crashed in handle_oneshot_fatal_signal(int)()

Please advise if any additional tests or information would be helpful.