Bug #64512
closedcrimson: asan stack-use-after-return false positive on osd startup with clang-17
0%
Description
On clang-17 (output below) and also gcc-12/13, address sanitizer seems to be throwing stack-use-after-return errors related to seastar::async:
INFO 2024-02-21 00:12:18,900 [shard 0:main] ms - [0x511000042800 client.?(temp_mon_client) 10.0.2.100:0/374772225@59765 >> mon.0 v2:10.0.2.100:40505/0] closing: reset no, replace no INFO 2024-02-21 00:12:18,901 [shard 0:main] ms - [0x511000042800 client.?(temp_mon_client) 10.0.2.100:0/374772225@59765 >> mon.0 v2:10.0.2.100:40505/0] do_in_dispatch(): fault at drop, io_stat(io_state=drop, in_ seq=3, out_seq=3, out_pending_msgs_size=0, out_sent_msgs_size=0, need_ack=0, need_keepalive=0, need_keepalive_ack=0) -- read eof ================================================================= ==123962==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f419f14dfa0 at pc 0x00000492b36e bp 0x7f419ef45c70 sp 0x7f419ef45c68 WRITE of size 8 at 0x7f419f14dfa0 thread T0 #0 0x492b36d in seastar::internal::future_base::future_base() /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/core/future.hh:1106:30 #1 0x4962978 in seastar::future<void>::future<>(seastar::ready_future_marker) /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/core/future.hh:1262:5 #2 0x49628f3 in seastar::future<void> seastar::make_ready_future<void>() /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/core/future.hh:1945:12 #3 0x4d7bc65 in seastar::future<void> seastar::futurize<void>::apply<crimson::osd::populate_config_from_mon()::$_0>(crimson::osd::populate_config_from_mon()::$_0&&, std::tuple<>&&) /home/sam/git-checkouts/cep h-workspace/main/src/seastar/include/seastar/core/future.hh:1999:20 #4 0x4d7bb18 in seastar::futurize<std::invoke_result<crimson::osd::populate_config_from_mon()::$_0>::type>::type seastar::async<crimson::osd::populate_config_from_mon()::$_0>(seastar::thread_attributes, crimson::osd::populate_config_from_mon()::$_0&&)::'lambda'()::operator()() const /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/core/thread.hh:260:13 #5 0x4d7b958 in seastar::noncopyable_function<void ()>::direct_vtable_for<seastar::futurize<std::invoke_result<crimson::osd::populate_config_from_mon()::$_0>::type>::type seastar::async<crimson::osd::populate_config_from_mon()::$_0>(seastar::thread_attributes, crimson::osd::populate_config_from_mon()::$_0&&)::'lambda'()>::call(seastar::noncopyable_function<void ()> const*) /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/util/noncopyable_function.hh:129:20 #6 0xf5ad4b2 in seastar::noncopyable_function<void ()>::operator()() const /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/util/noncopyable_function.hh:215:16 #7 0xfc7ebf7 in seastar::thread_context::main() /home/sam/git-checkouts/ceph-workspace/main/src/seastar/src/core/thread.cc:311:9 Address 0x7f419f14dfa0 is located in stack of thread T0 at offset 32 in frame #0 0x49749cf in seastar::internal::log_buf::inserter_iterator::operator++(int) /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/util/log-impl.hh:89 This frame has 1 object(s): [32, 40) 'retval' <== Memory access at offset 32 is inside this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-use-after-return /home/sam/git-checkouts/ceph-workspace/main/src/seastar/include/seastar/core/future.hh:1106:30 in seastar::internal::future_base::future_base() Shadow bytes around the buggy address:
The actual bug appears to be that seastar is getting built without Sanitizers_FIBER_SUPPORT. seastar has a cmake module which checks and sets it (.src/seastar/cmake/FindSanitizers.cmake), but there appears to be no machinery that actually invokes it (no include() or find_package()). ceph also has a FindSanitizers.cmake with similar goals (./cmake/modules/FindSanitizers.cmake) but which doesn't include set Sanitizers_FIBER_SUPPORT. I'm preparing a fix to copy that part over.
See src/seastar/src/core/thread.cc for the SEASTAR_HAVE_ASAN_FIBER_SUPPORT usage and src/seastar/CMakeLists.txt for the Sanitizers_FIBER_SUPPORT check:
if (Sanitizers_FIBER_SUPPORT) list (APPEND Seastar_PRIVATE_COMPILE_DEFINITIONS SEASTAR_HAVE_ASAN_FIBER_SUPPORT) endif ()
I'm not actually sure why this wasn't triggering on older clang and gcc versions -- it likely just wasn't on by default.