Project

General

Profile

Actions
Copy link

Bug #62934

open

unittest_osdmap (Subprocess aborted) during OSDMapTest.BUG_42485

Added by Casey Bodley 8 months ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

full log: https://jenkins.ceph.com/job/ceph-pull-requests/122260/consoleText

205/283 Test #194: unittest_osdmap ...........................Subprocess aborted***Exception:  41.07 sec
[==========] Running 32 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 27 tests from OSDMapTest
[ RUN      ] OSDMapTest.Create
[       OK ] OSDMapTest.Create (1 ms)
[ RUN      ] OSDMapTest.Features
[       OK ] OSDMapTest.Features (1 ms)
[ RUN      ] OSDMapTest.MapPG
 osdmap.pool_max==2
[       OK ] OSDMapTest.MapPG (1 ms)
[ RUN      ] OSDMapTest.MapFunctionsMatch
[       OK ] OSDMapTest.MapFunctionsMatch (0 ms)
[ RUN      ] OSDMapTest.PrimaryIsFirst
[       OK ] OSDMapTest.PrimaryIsFirst (1 ms)
[ RUN      ] OSDMapTest.PGTempRespected
[       OK ] OSDMapTest.PGTempRespected (1 ms)
[ RUN      ] OSDMapTest.PrimaryTempRespected
[       OK ] OSDMapTest.PrimaryTempRespected (0 ms)
[ RUN      ] OSDMapTest.CleanTemps
[       OK ] OSDMapTest.CleanTemps (1 ms)
[ RUN      ] OSDMapTest.KeepsNecessaryTemps
[       OK ] OSDMapTest.KeepsNecessaryTemps (1 ms)
[ RUN      ] OSDMapTest.PrimaryAffinity
pool 1 size 3 expect_primary 1666
any: [6095,5782,4528,5310,5003,3282]
first: [1875,1564,1562,2655,1406,938]
primary: [1875,1564,1562,2655,1406,938]
any: [6095,5782,4528,5310,5003,3282]
first: [1875,1564,1562,2655,1406,938]
primary: [0,0,2967,2968,2501,1564]
any: [6095,5782,4528,5310,5003,3282]
first: [1875,1564,1562,2655,1406,938]
primary: [938,0,2499,2812,2344,1407]
expect 1250
pool 2 size 3 expect_primary 1666
any: [5939,5937,3751,4374,4529,5470]
first: [1874,2031,1251,1408,1093,2343]
primary: [1874,2031,1251,1408,1093,2343]
any: [5939,5937,3751,4374,4529,5470]
first: [0,0,2033,2188,2498,3281]
primary: [0,0,2033,2188,2498,3281]
any: [5939,5937,3751,4374,4529,5470]
first: [1250,0,1720,1876,2186,2968]
primary: [1250,0,1720,1876,2186,2968]
expect 1250
[       OK ] OSDMapTest.PrimaryAffinity (870 ms)
[ RUN      ] OSDMapTest.get_osd_crush_node_flags
[       OK ] OSDMapTest.get_osd_crush_node_flags (1 ms)
[ RUN      ] OSDMapTest.parse_osd_id_list
Expected option value to be integer, got 'foo'invalid osd id 'foo'expected numerical value, got: -12invalid osd id '-12'[       OK ] OSDMapTest.parse_osd_id_list (0 ms)
[ RUN      ] OSDMapTest.CleanPGUpmaps
2023-09-21T17:58:56.370+0000 7f7689961640 -1 verify_upmap multiple osds 2,3 come from same failure domain -5
2023-09-21T17:58:56.370+0000 7f7689961640  0 check_pg_upmaps verify_upmap of pg 3.0 returning -22
2023-09-21T17:58:56.374+0000 7f7689160640 -1 verify_upmap multiple osds 4,5 come from same failure domain -6
2023-09-21T17:58:56.374+0000 7f7689160640  0 check_pg_upmaps verify_upmap of pg 3.0 returning -22
[       OK ] OSDMapTest.CleanPGUpmaps (26 ms)
[ RUN      ] OSDMapTest.BUG_38897
[       OK ] OSDMapTest.BUG_38897 (5 ms)
[ RUN      ] OSDMapTest.BUG_40104
clean_pg_upmaps (~10000 pg_upmap_items) latency:6s
[       OK ] OSDMapTest.BUG_40104 (39942 ms)
[ RUN      ] OSDMapTest.BUG_42052
[       OK ] OSDMapTest.BUG_42052 (3 ms)
[ RUN      ] OSDMapTest.BUG_42485
pgid [26,12,54,46] up [26,12,54,46]
from 26 to 30
pgid 3.2 up [16,25,53,31]
from 16 to 40
from 31 to 0
pure virtual method called
terminate called without an active exception
*** Caught signal (Aborted) **
 in thread 7f768895f640 thread_name:clean_upmap_tp
 ceph version Development (no_version) reef (stable)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2d70f3) [0x5620cad660f3]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f76919f9520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bfe) [0x7f7691d8abfe]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7f7691d9628c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7f7691d962f7]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaf025) [0x7f7691d97025]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x5620cacb65d6]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7f76939c54a5]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7f76939cb2fa]
 13: (Thread::entry_wrapper()+0x84) [0x7f769399af14]
 14: (Thread::_entry_func(void*)+0x15) [0x7f769399ae75]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f7691a4bb43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f7691adda00]
2023-09-21T17:59:36.383+0000 7f768895f640 -1 *** Caught signal (Aborted) **
 in thread 7f768895f640 thread_name:clean_upmap_tp

 ceph version Development (no_version) reef (stable)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2d70f3) [0x5620cad660f3]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f76919f9520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bfe) [0x7f7691d8abfe]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7f7691d9628c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7f7691d962f7]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaf025) [0x7f7691d97025]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x5620cacb65d6]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7f76939c54a5]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7f76939cb2fa]
 13: (Thread::entry_wrapper()+0x84) [0x7f769399af14]
 14: (Thread::_entry_func(void*)+0x15) [0x7f769399ae75]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f7691a4bb43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f7691adda00]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
   -94> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command assert hook 0x5620cb15e600
   -93> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command abort hook 0x5620cb15e600
   -92> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command leak_some_memory hook 0x5620cb15e600
   -91> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perfcounters_dump hook 0x5620cb15e600
   -90> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command 1 hook 0x5620cb15e600
   -89> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf dump hook 0x5620cb15e600
   -88> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perfcounters_schema hook 0x5620cb15e600
   -87> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf histogram dump hook 0x5620cb15e600
   -86> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command 2 hook 0x5620cb15e600
   -85> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf schema hook 0x5620cb15e600
   -84> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command counter dump hook 0x5620cb15e600
   -83> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command counter schema hook 0x5620cb15e600
   -82> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf histogram schema hook 0x5620cb15e600
   -81> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf reset hook 0x5620cb15e600
   -80> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config show hook 0x5620cb15e600
   -79> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config help hook 0x5620cb15e600
   -78> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config set hook 0x5620cb15e600
   -77> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config unset hook 0x5620cb15e600
   -76> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config get hook 0x5620cb15e600
   -75> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config diff hook 0x5620cb15e600
   -74> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config diff get hook 0x5620cb15e600
   -73> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command injectargs hook 0x5620cb15e600
   -72> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command log flush hook 0x5620cb15e600
   -71> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command log dump hook 0x5620cb15e600
   -70> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command log reopen hook 0x5620cb15e600
   -69> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command dump_mempools hook 0x5620cb22f048
   -68> 2023-09-21T17:58:56.358+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -67> 2023-09-21T17:58:56.358+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -66> 2023-09-21T17:58:56.358+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -65> 2023-09-21T17:58:56.358+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -64> 2023-09-21T17:58:56.358+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -63> 2023-09-21T17:58:56.358+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -62> 2023-09-21T17:58:56.358+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -61> 2023-09-21T17:58:56.362+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -60> 2023-09-21T17:58:56.362+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -59> 2023-09-21T17:58:56.362+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -58> 2023-09-21T17:58:56.362+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -57> 2023-09-21T17:58:56.362+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -56> 2023-09-21T17:58:56.362+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -55> 2023-09-21T17:58:56.362+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -54> 2023-09-21T17:58:56.362+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -53> 2023-09-21T17:58:56.362+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -52> 2023-09-21T17:58:56.362+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -51> 2023-09-21T17:58:56.362+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -50> 2023-09-21T17:58:56.362+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -49> 2023-09-21T17:58:56.362+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -48> 2023-09-21T17:58:56.362+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -47> 2023-09-21T17:58:56.362+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -46> 2023-09-21T17:58:56.362+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -45> 2023-09-21T17:58:56.366+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -44> 2023-09-21T17:58:56.370+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -43> 2023-09-21T17:58:56.370+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -42> 2023-09-21T17:58:56.370+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -41> 2023-09-21T17:58:56.370+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -40> 2023-09-21T17:58:56.370+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -39> 2023-09-21T17:58:56.370+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -38> 2023-09-21T17:58:56.370+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -37> 2023-09-21T17:58:56.370+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -36> 2023-09-21T17:58:56.370+0000 7f7689961640 -1 verify_upmap multiple osds 2,3 come from same failure domain -5
   -35> 2023-09-21T17:58:56.370+0000 7f7689961640  0 check_pg_upmaps verify_upmap of pg 3.0 returning -22
   -34> 2023-09-21T17:58:56.370+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -33> 2023-09-21T17:58:56.370+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -32> 2023-09-21T17:58:56.370+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -31> 2023-09-21T17:58:56.370+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -30> 2023-09-21T17:58:56.370+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -29> 2023-09-21T17:58:56.370+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -28> 2023-09-21T17:58:56.370+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -27> 2023-09-21T17:58:56.370+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -26> 2023-09-21T17:58:56.374+0000 7f7689160640 -1 verify_upmap multiple osds 4,5 come from same failure domain -6
   -25> 2023-09-21T17:58:56.374+0000 7f7689160640  0 check_pg_upmaps verify_upmap of pg 3.0 returning -22
   -24> 2023-09-21T17:58:56.374+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -23> 2023-09-21T17:58:56.374+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -22> 2023-09-21T17:58:56.374+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -21> 2023-09-21T17:58:56.374+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -20> 2023-09-21T17:58:56.374+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -19> 2023-09-21T17:58:56.378+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -18> 2023-09-21T17:58:56.378+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -17> 2023-09-21T17:58:56.378+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -16> 2023-09-21T17:59:36.307+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -15> 2023-09-21T17:59:36.307+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -14> 2023-09-21T17:59:36.307+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -13> 2023-09-21T17:59:36.307+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -12> 2023-09-21T17:59:36.307+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -11> 2023-09-21T17:59:36.307+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -10> 2023-09-21T17:59:36.307+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
    -9> 2023-09-21T17:59:36.307+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
    -8> 2023-09-21T17:59:36.323+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
    -7> 2023-09-21T17:59:36.323+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
    -6> 2023-09-21T17:59:36.323+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
    -5> 2023-09-21T17:59:36.323+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
    -4> 2023-09-21T17:59:36.323+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
    -3> 2023-09-21T17:59:36.323+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
    -2> 2023-09-21T17:59:36.323+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
    -1> 2023-09-21T17:59:36.327+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
     0> 2023-09-21T17:59:36.383+0000 7f768895f640 -1 *** Caught signal (Aborted) **
 in thread 7f768895f640 thread_name:clean_upmap_tp

 ceph version Development (no_version) reef (stable)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2d70f3) [0x5620cad660f3]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f76919f9520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bfe) [0x7f7691d8abfe]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7f7691d9628c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7f7691d962f7]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaf025) [0x7f7691d97025]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x5620cacb65d6]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7f76939c54a5]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7f76939cb2fa]
 13: (Thread::entry_wrapper()+0x84) [0x7f769399af14]
 14: (Thread::_entry_func(void*)+0x15) [0x7f769399ae75]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f7691a4bb43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f7691adda00]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 rbd_mirror
   0/ 5 rbd_replay
   0/ 5 rbd_pwl
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 immutable_obj_cache
   0/ 5 client
   1/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 0 ms
   1/ 5 mon
   0/10 monc
   1/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 1 reserver
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 rgw_sync
   1/ 5 rgw_datacache
   1/ 5 rgw_access
   1/ 5 rgw_dbstore
   1/ 5 rgw_flight
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
   0/ 0 refs
   1/ 5 compressor
   1/ 5 bluestore
   1/ 5 bluefs
   1/ 3 bdev
   1/ 5 kstore
   4/ 5 rocksdb
   4/ 5 leveldb
   1/ 5 fuse
   2/ 5 mgr
   1/ 5 mgrc
   1/ 5 dpdk
   1/ 5 eventtrace
   1/ 5 prioritycache
   0/ 5 test
   0/ 5 cephfs_mirror
   0/ 5 cephsqlite
   0/ 5 seastore
   0/ 5 seastore_onode
   0/ 5 seastore_odata
   0/ 5 seastore_omap
   0/ 5 seastore_tm
   0/ 5 seastore_t
   0/ 5 seastore_cleaner
   0/ 5 seastore_epm
   0/ 5 seastore_lba
   0/ 5 seastore_fixedkv_tree
   0/ 5 seastore_cache
   0/ 5 seastore_journal
   0/ 5 seastore_device
   0/ 5 seastore_backref
   0/ 5 alienstore
   1/ 5 mclock
   0/ 5 cyanstore
   1/ 5 ceph_exporter
   1/ 5 memstore
  -2/-2 (syslog threshold)
  99/99 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
  7f768815e640 / clean_upmap_tp
  7f768895f640 / clean_upmap_tp
  7f7689160640 / clean_upmap_tp
  7f7689961640 / clean_upmap_tp
  7f768a162640 / clean_upmap_tp
  7f768a963640 / clean_upmap_tp
  7f768b164640 / clean_upmap_tp
  7f768b965640 / clean_upmap_tp
  7f7690e1e2c0 / unittest_osdmap
  max_recent       500
  max_new         1000
  log_file 
--- end dump of recent events ---
--- begin dump of recent events ---
   -94> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command assert hook 0x5620cb15e600
   -93> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command abort hook 0x5620cb15e600
   -92> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command leak_some_memory hook 0x5620cb15e600
   -91> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perfcounters_dump hook 0x5620cb15e600
   -90> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command 1 hook 0x5620cb15e600
   -89> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf dump hook 0x5620cb15e600
   -88> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perfcounters_schema hook 0x5620cb15e600
   -87> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf histogram dump hook 0x5620cb15e600
   -86> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command 2 hook 0x5620cb15e600
   -85> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf schema hook 0x5620cb15e600
   -84> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command counter dump hook 0x5620cb15e600
   -83> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command counter schema hook 0x5620cb15e600
   -82> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf histogram schema hook 0x5620cb15e600
   -81> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command perf reset hook 0x5620cb15e600
   -80> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config show hook 0x5620cb15e600
   -79> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config help hook 0x5620cb15e600
   -78> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config set hook 0x5620cb15e600
   -77> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config unset hook 0x5620cb15e600
   -76> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config get hook 0x5620cb15e600
   -75> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config diff hook 0x5620cb15e600
   -74> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command config diff get hook 0x5620cb15e600
   -73> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command injectargs hook 0x5620cb15e600
   -72> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command log flush hook 0x5620cb15e600
   -71> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command log dump hook 0x5620cb15e600
   -70> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command log reopen hook 0x5620cb15e600
   -69> 2023-09-21T17:58:55.458+0000 7f7690e1e2c0  5 asok(0x5620cb19e740) register_command dump_mempools hook 0x5620cb22f048
   -68> 2023-09-21T17:58:56.358+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -67> 2023-09-21T17:58:56.358+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -66> 2023-09-21T17:58:56.358+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -65> 2023-09-21T17:58:56.358+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -64> 2023-09-21T17:58:56.358+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -63> 2023-09-21T17:58:56.358+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -62> 2023-09-21T17:58:56.358+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -61> 2023-09-21T17:58:56.362+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -60> 2023-09-21T17:58:56.362+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -59> 2023-09-21T17:58:56.362+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -58> 2023-09-21T17:58:56.362+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -57> 2023-09-21T17:58:56.362+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -56> 2023-09-21T17:58:56.362+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -55> 2023-09-21T17:58:56.362+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -54> 2023-09-21T17:58:56.362+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -53> 2023-09-21T17:58:56.362+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -52> 2023-09-21T17:58:56.362+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -51> 2023-09-21T17:58:56.362+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -50> 2023-09-21T17:58:56.362+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -49> 2023-09-21T17:58:56.362+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -48> 2023-09-21T17:58:56.362+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -47> 2023-09-21T17:58:56.362+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -46> 2023-09-21T17:58:56.362+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -45> 2023-09-21T17:58:56.366+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -44> 2023-09-21T17:58:56.370+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -43> 2023-09-21T17:58:56.370+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -42> 2023-09-21T17:58:56.370+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -41> 2023-09-21T17:58:56.370+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -40> 2023-09-21T17:58:56.370+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -39> 2023-09-21T17:58:56.370+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -38> 2023-09-21T17:58:56.370+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -37> 2023-09-21T17:58:56.370+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -36> 2023-09-21T17:58:56.370+0000 7f7689961640 -1 verify_upmap multiple osds 2,3 come from same failure domain -5
   -35> 2023-09-21T17:58:56.370+0000 7f7689961640  0 check_pg_upmaps verify_upmap of pg 3.0 returning -22
   -34> 2023-09-21T17:58:56.370+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -33> 2023-09-21T17:58:56.370+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -32> 2023-09-21T17:58:56.370+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -31> 2023-09-21T17:58:56.370+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -30> 2023-09-21T17:58:56.370+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -29> 2023-09-21T17:58:56.370+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -28> 2023-09-21T17:58:56.370+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -27> 2023-09-21T17:58:56.370+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -26> 2023-09-21T17:58:56.374+0000 7f7689160640 -1 verify_upmap multiple osds 4,5 come from same failure domain -6
   -25> 2023-09-21T17:58:56.374+0000 7f7689160640  0 check_pg_upmaps verify_upmap of pg 3.0 returning -22
   -24> 2023-09-21T17:58:56.374+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -23> 2023-09-21T17:58:56.374+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
   -22> 2023-09-21T17:58:56.374+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
   -21> 2023-09-21T17:58:56.374+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -20> 2023-09-21T17:58:56.374+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -19> 2023-09-21T17:58:56.378+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -18> 2023-09-21T17:58:56.378+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -17> 2023-09-21T17:58:56.378+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -16> 2023-09-21T17:59:36.307+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
   -15> 2023-09-21T17:59:36.307+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
   -14> 2023-09-21T17:59:36.307+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
   -13> 2023-09-21T17:59:36.307+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
   -12> 2023-09-21T17:59:36.307+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
   -11> 2023-09-21T17:59:36.307+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
   -10> 2023-09-21T17:59:36.307+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
    -9> 2023-09-21T17:59:36.307+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
    -8> 2023-09-21T17:59:36.323+0000 7f768815e640  1 BUG_40104::clean_upmap_tp worker finish
    -7> 2023-09-21T17:59:36.323+0000 7f768895f640  1 BUG_40104::clean_upmap_tp worker finish
    -6> 2023-09-21T17:59:36.323+0000 7f768a963640  1 BUG_40104::clean_upmap_tp worker finish
    -5> 2023-09-21T17:59:36.323+0000 7f768a162640  1 BUG_40104::clean_upmap_tp worker finish
    -4> 2023-09-21T17:59:36.323+0000 7f768b965640  1 BUG_40104::clean_upmap_tp worker finish
    -3> 2023-09-21T17:59:36.323+0000 7f768b164640  1 BUG_40104::clean_upmap_tp worker finish
    -2> 2023-09-21T17:59:36.323+0000 7f7689961640  1 BUG_40104::clean_upmap_tp worker finish
    -1> 2023-09-21T17:59:36.327+0000 7f7689160640  1 BUG_40104::clean_upmap_tp worker finish
     0> 2023-09-21T17:59:36.383+0000 7f768895f640 -1 *** Caught signal (Aborted) **
 in thread 7f768895f640 thread_name:clean_upmap_tp

 ceph version Development (no_version) reef (stable)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2d70f3) [0x5620cad660f3]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f76919f9520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bfe) [0x7f7691d8abfe]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7f7691d9628c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7f7691d962f7]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaf025) [0x7f7691d97025]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x5620cacb65d6]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7f76939c54a5]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7f76939cb2fa]
 13: (Thread::entry_wrapper()+0x84) [0x7f769399af14]
 14: (Thread::_entry_func(void*)+0x15) [0x7f769399ae75]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f7691a4bb43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f7691adda00]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 rbd_mirror
   0/ 5 rbd_replay
   0/ 5 rbd_pwl
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 immutable_obj_cache
   0/ 5 client
   1/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 0 ms
   1/ 5 mon
   0/10 monc
   1/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 1 reserver
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 rgw_sync
   1/ 5 rgw_datacache
   1/ 5 rgw_access
   1/ 5 rgw_dbstore
   1/ 5 rgw_flight
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
   0/ 0 refs
   1/ 5 compressor
   1/ 5 bluestore
   1/ 5 bluefs
   1/ 3 bdev
   1/ 5 kstore
   4/ 5 rocksdb
   4/ 5 leveldb
   1/ 5 fuse
   2/ 5 mgr
   1/ 5 mgrc
   1/ 5 dpdk
   1/ 5 eventtrace
   1/ 5 prioritycache
   0/ 5 test
   0/ 5 cephfs_mirror
   0/ 5 cephsqlite
   0/ 5 seastore
   0/ 5 seastore_onode
   0/ 5 seastore_odata
   0/ 5 seastore_omap
   0/ 5 seastore_tm
   0/ 5 seastore_t
   0/ 5 seastore_cleaner
   0/ 5 seastore_epm
   0/ 5 seastore_lba
   0/ 5 seastore_fixedkv_tree
   0/ 5 seastore_cache
   0/ 5 seastore_journal
   0/ 5 seastore_device
   0/ 5 seastore_backref
   0/ 5 alienstore
   1/ 5 mclock
   0/ 5 cyanstore
   1/ 5 ceph_exporter
   1/ 5 memstore
  -2/-2 (syslog threshold)
  99/99 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
  7f768815e640 / clean_upmap_tp
  7f768895f640 / clean_upmap_tp
  7f7689160640 / clean_upmap_tp
  7f7689961640 / clean_upmap_tp
  7f768a162640 / clean_upmap_tp
  7f768a963640 / clean_upmap_tp
  7f768b164640 / clean_upmap_tp
  7f768b965640 / clean_upmap_tp
  7f7690e1e2c0 / unittest_osdmap
  max_recent       500
  max_new         1000
  log_file /var/lib/ceph/crash/2023-09-21T17:59:36.389680Z_e153b1b5-4873-42b7-8c2e-35a7b48604d5/log
--- end dump of recent events ---

Actions
Copy link
 #1

Updated by Radoslaw Zarzynski 7 months ago

The evidence from Neha it's not a recent thing -- is was seen in 2021: https://github.com/ceph/ceph/pull/41848#issuecomment-862208913.

Actions
Copy link
 #3

Updated by Radoslaw Zarzynski 6 months ago

The patch got merged pretty recently, in 44th week of 2023. Let's observe!

Actions
Copy link
 #4

Updated by Casey Bodley 4 months ago

still seeing these on main:

The following tests FAILED:
    198 - unittest_osdmap (Subprocess aborted)

ex. https://jenkins.ceph.com/job/ceph-pull-requests/127638/testReport/junit/projectroot.src.test/osd/unittest_osdmap/
    -1> 2024-01-16T16:07:48.078+0000 7f6baa7a5640  1 BUG_40104::clean_upmap_tp worker finish
     0> 2024-01-16T16:07:48.086+0000 7f6ba6f9e640 -1 *** Caught signal (Aborted) **
 in thread 7f6ba6f9e640 thread_name:clean_upmap_tp

 ceph version Development (no_version) squid (dev)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2d7ac2) [0x5564deab2ac2]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6bb0834520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa4f26) [0x7f6bb0bcbf26]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xb6d9c) [0x7f6bb0bddd9c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xb6e07) [0x7f6bb0bdde07]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xb7b35) [0x7f6bb0bdeb35]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x5564de9ff326]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7f6bb2996635]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7f6bb299c63a]
 13: (Thread::entry_wrapper()+0x84) [0x7f6bb296ac84]
 14: (Thread::_entry_func(void*)+0x15) [0x7f6bb296abe5]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6bb0886b43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6bb0918a00]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Actions
Copy link
 #5

Updated by Kefu Chai 3 months ago

OSDMapTest.BUG_43124 this time

[ RUN      ] OSDMapTest.BUG_43124
ID   CLASS  WEIGHT     TYPE NAME             
 -1         200.00000  root default          
...
pgid 3.0 up [54,70,63,117,88,106,148,124,138,26,6,11]
from 54 to 30
pure virtual method called
terminate called without an active exception
*** Caught signal (Aborted) **
 in thread 7fa2a0156640 thread_name:clean_upmap_tp
 ceph version Development (no_version) squid (dev)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2ed41e) [0x55af41e1d41e]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa2a71e9520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bfe) [0x7fa2a7578bfe]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7fa2a758428c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7fa2a75842f7]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaf025) [0x7fa2a7585025]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x55af41d64d46]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7fa2a93092f5]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7fa2a930f2fa]
 13: (Thread::entry_wrapper()+0x84) [0x7fa2a92dd984]
 14: (Thread::_entry_func(void*)+0x15) [0x7fa2a92dd8e5]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fa2a723bb43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fa2a72cda00]
2024-02-08T16:03:41.903+0000 7fa2a0156640 -1 *** Caught signal (Aborted) **
 in thread 7fa2a0156640 thread_name:clean_upmap_tp

 ceph version Development (no_version) squid (dev)
 1: /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_osdmap(+0x2ed41e) [0x55af41e1d41e]
 2: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa2a71e9520]
 3: pthread_kill()
 4: raise()
 5: abort()
 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bfe) [0x7fa2a7578bfe]
 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae28c) [0x7fa2a758428c]
 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2f7) [0x7fa2a75842f7]
 9: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaf025) [0x7fa2a7585025]
 10: (ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue()+0x16) [0x55af41d64d46]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0x7a5) [0x7fa2a93092f5]
 12: (ThreadPool::WorkThread::entry()+0x1a) [0x7fa2a930f2fa]
 13: (Thread::entry_wrapper()+0x84) [0x7fa2a92dd984]
 14: (Thread::_entry_func(void*)+0x15) [0x7fa2a92dd8e5]
 15: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fa2a723bb43]
 16: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fa2a72cda00]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

see https://jenkins.ceph.com/job/ceph-pull-requests/128930/testReport/junit/projectroot.src.test/osd/unittest_osdmap/

Actions
Copy link
 #6

Updated by Radoslaw Zarzynski 3 months ago

The tested PR https://github.com/ceph/ceph/pull/55428 (main), so it's clearly a different thing that https://tracker.ceph.com/issues/63310 & mentioned earlier.

Actions
Copy link
 #7

Updated by Radoslaw Zarzynski 3 months ago

note from scrub: Nitzan is adding more context. In short: this might be a test issue,

Actions
Copy link
 #8

Updated by Nitzan Mordechai 3 months ago

since all of them getting "pure virtual method called" i don't think it's related to osdmap, but to the ThreadPool work queue that getting the wrong value in some way.

Actions
Copy link

Also available in: Atom PDF