Project

General

Profile

Bug #56456 » osd-crash-dmesg.log

dmesg from machine kw210 (i.e. kube-worker 210) - Sébastien Bernard, 07/04/2022 07:56 PM

 
[2022-06-29T16:21:30.783632326Z]: show_signal_msg: 13 callbacks suppressed
[2022-06-29T16:21:30.783639326Z]: bstore_kv_final[44116]: segfault at 559d00000001 ip 00007feec218c573 sp 00007feeb1ba7fa0 error 4 in libtcmalloc.so.4.5.3[7feec2161000+4d000]
[2022-06-29T16:21:31.053922326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-06-29T16:21:31.212756326Z]: msgr-worker-2[37118]: segfault at 559d00000000 ip 0000559d8ae7052b sp 00007feebc64f3b0 error 4
[2022-06-29T16:21:31.214046326Z]: msgr-worker-1[37117]: segfault at 559d8be4d118 ip 0000559d8b2d3e59 sp 00007feebce50ac0 error 7 in ceph-osd[559d8a400000+1813000]
[2022-06-29T16:21:31.214071326Z]: Code: ee 74 48 49 89 fc 49 89 f5 0f 1f 84 00 00 00 00 00 bf 18 00 00 00 e8 b6 8b 64 ff 48 8d 75 08 48 89 c3 48 8b 45 00 48 8d 7b 08 <48> 89 03 e8 df e0 ff ff 49 8b 44 24 08 4c 89 23 48 89 18 49 89 5c
[2022-06-29T16:21:31.877058326Z]: in ceph-osd[559d8a400000+1813000]
[2022-06-29T16:21:31.946032326Z]: Code: 08 4c 8b 2f 49 39 fd 74 5b 49 89 ff 0f 1f 00 49 8b 5d 18 4d 8d 65 18 4d 8b 75 00 49 39 dc 74 34 0f 1f 80 00 00 00 00 48 89 df <48> 8b 2b e8 ad 36 46 00 84 c0 75 16 48 8d 7b 08 e8 50 1b 46 00 be
[2022-06-29T19:23:13.099953326Z]: bstore_kv_final[44930]: segfault at 0 ip 00007fb1b833357b sp 00007fb1a7d4f030 error 4 in libtcmalloc.so.4.5.3[7fb1b8308000+4d000]
[2022-06-29T19:23:13.281163326Z]: Code: 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 48 8b 12 41 39 c7 75 f5 <48> 8b 02 48 89 45 00 48 c7 02 00 00 00 00 8b 45 08 44 29 f8 89 45
[2022-06-29T22:22:30.825688326Z]: tp_osd_tp[44538]: segfault at 0 ip 00007fefc5d85573 sp 00007fefa414a160 error 4 in libtcmalloc.so.4.5.3[7fefc5d5a000+4d000]
[2022-06-29T22:22:30.850821326Z]: traps: bstore_kv_final[44348] general protection fault ip:7fefc5d85573 sp:7fefb57a0eb0 error:0
[2022-06-29T22:22:31.000199326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-06-29T22:22:31.384663326Z]: in libtcmalloc.so.4.5.3[7fefc5d5a000+4d000]
[2022-06-30T03:57:38.299317326Z]: bstore_kv_final[43437]: segfault at 558200000001 ip 00007fac65076d6a sp 00007fac54a81ff8 error 4 in libtcmalloc.so.4.5.3[7fac6503b000+4d000]
[2022-06-30T03:57:38.490745326Z]: Code: 34 02 89 f0 48 89 f1 4c 63 8c 82 fc 09 00 00 4d 29 88 08 0c 00 00 4c 89 ca 78 35 48 c1 e1 05 4c 01 c1 48 8b 01 48 85 c0 74 4e <48> 8b 30 8b 79 08 48 89 31 8d 77 ff 89 71 08 3b 71 0c 72 32 41 29
[2022-06-30T15:43:24.010936326Z]: traps: tp_osd_tp[44044] general protection fault ip:7fc75f707d6a sp:7fc737aaf5d8 error:0 in libtcmalloc.so.4.5.3[7fc75f6cc000+4d000]
[2022-06-30T15:43:24.096008326Z]: traps: tp_osd_tp[44045] general protection fault ip:7fc75ebbbf58 sp:7fc7372a81d0 error:0 in libpthread-2.28.so[7fc75eba8000+1b000]
[2022-06-30T19:27:25.761884326Z]: tp_osd_tp[43544]: segfault at 0 ip 00007ff8f1376573 sp 00007ff8cf7399e0 error 4 in libtcmalloc.so.4.5.3[7ff8f134b000+4d000]
[2022-06-30T19:27:25.935266326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-07-01T21:07:45.231746326Z]: traps: msgr-worker-1[155652] general protection fault ip:7f0798aa4d6a sp:7f0793758ab8 error:0 in libtcmalloc.so.4.5.3[7f0798a69000+4d000]
[2022-07-02T03:40:18.758755326Z]: tp_osd_tp[217731]: segfault at 0 ip 00007fa6402f4573 sp 00007fa61eeb17e0 error 4 in libtcmalloc.so.4.5.3[7fa6402c9000+4d000]
[2022-07-02T03:40:18.770206326Z]: traps: bstore_kv_final[217636] general protection fault ip:7fa6402f4573 sp:7fa62fd16f90 error:0 in libtcmalloc.so.4.5.3[7fa6402c9000+4d000]
[2022-07-02T03:40:18.932693326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-07-02T08:11:05.004571326Z]: bstore_kv_final[57069]: segfault at 0 ip 00007f7defd5a573 sp 00007f7ddf774bd0 error 4 in libtcmalloc.so.4.5.3[7f7defd2f000+4d000]
[2022-07-02T08:11:05.183746326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-07-02T08:11:30.458641326Z]: bstore_kv_final[44386]: segfault at 0 ip 00007f7f54a56573 sp 00007f7f44478ef0 error 4 in libtcmalloc.so.4.5.3[7f7f54a2b000+4d000]
[2022-07-02T08:11:30.638089326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-07-02T09:33:46.491397326Z]: bstore_kv_final[44342]: segfault at 0 ip 00007fe8a5e0e573 sp 00007fe895829f20 error 4 in libtcmalloc.so.4.5.3[7fe8a5de3000+4d000]
[2022-07-02T09:33:46.671135326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-07-02T12:49:34.711621326Z]: bstore_kv_final[56762]: segfault at 55fe00000001 ip 00007fed5ff23d6a sp 00007fed4f92ee38 error 4 in libtcmalloc.so.4.5.3[7fed5fee8000+4d000]
[2022-07-02T12:49:34.902933326Z]: Code: 34 02 89 f0 48 89 f1 4c 63 8c 82 fc 09 00 00 4d 29 88 08 0c 00 00 4c 89 ca 78 35 48 c1 e1 05 4c 01 c1 48 8b 01 48 85 c0 74 4e <48> 8b 30 8b 79 08 48 89 31 8d 77 ff 89 71 08 3b 71 0c 72 32 41 29
[2022-07-02T13:45:27.216489326Z]: traps: msgr-worker-2[35372] general protection fault ip:7f498e172d6a sp:7f498861e3d8 error:0 in libtcmalloc.so.4.5.3[7f498e137000+4d000]
[2022-07-02T16:15:01.752759326Z]: bstore_kv_final[42565]: segfault at 0 ip 00007fbcffc4a573 sp 00007fbcef665f60 error 4 in libtcmalloc.so.4.5.3[7fbcffc1f000+4d000]
[2022-07-02T16:15:01.824839326Z]: traps: tp_osd_tp[42621] general protection fault ip:564fa536b30d sp:7fbcd8002ff0 error:0 in ceph-osd[564fa4800000+1813000]
[2022-07-02T16:15:01.932822326Z]: Code: d9 4c 89 e7 41 29 df e8 4b c0 fe ff 44 39 fb 7c a6 48 8b 75 00 48 89 f2 41 83 ff 01 7e 14 b8 01 00 00 00 0f 1f 40 00 83 c0 01 <48> 8b 12 41 39 c7 75 f5 48 8b 02 48 89 45 00 48 c7 02 00 00 00 00
[2022-07-03T14:25:48.381241326Z]: hrtimer: interrupt took 114284 ns
[2022-07-04T08:05:40.226549326Z]: traps: tp_osd_tp[56897] general protection fault ip:7f8790733d6a sp:7f87682db028 error:0 in libtcmalloc.so.4.5.3[7f87906f8000+4d000]
[2022-07-04T12:10:14.565158326Z]: traps: bstore_kv_final[51824] general protection fault ip:7f84b7a40573 sp:7f84a745bf60 error:0 in libtcmalloc.so.4.5.3[7f84b7a15000+4d000]

(1-1/7)