Actions
Bug #36405
closedunittest_seastar_messenger failure on ARM
Added by John Spray over 5 years ago. Updated almost 5 years ago.
Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Correctness/Safety
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
We often ignore these failures, but when I looked at the log I realised it's actually a recently added test that's failing, so probably a genuine bug?
https://jenkins.ceph.com/job/ceph-pull-requests-arm64/24113/console
The following tests FAILED:
157 - unittest_seastar_messenger (SEGFAULT)
Updated by Kefu Chai over 5 years ago
from dmesg:
[19321510.804024] unittest_seasta[46470]: unhandled level 3 permission fault (11) at 0x3284d0e0, esr 0x8300000f [19321510.804032] pgd = ffff80048b1df000 [19321510.811872] [3284d0e0] *pgd=00000002e4566003, *pud=00000006b3335003, *pmd=000000038da8b003, *pte=00e8000de49c5f53 [19321510.826681] CPU: 0 PID: 46470 Comm: unittest_seasta Not tainted 4.4.0-121-generic #145-Ubuntu [19321510.826684] Hardware name: GIGABYTE R120-T32-00/MT30-GS1-00, BIOS T32 03/03/2017 [19321510.826688] task: ffff800f8315ce00 ti: ffff8001d8a74000 task.ti: ffff8001d8a74000 [19321510.826693] PC is at 0x3284d0e0 [19321510.826696] LR is at 0x988588 [19321510.826699] pc : [<000000003284d0e0>] lr : [<0000000000988588>] pstate: 20000000 [19321510.826702] sp : 0000ffffdb7a2f40 [19321510.826704] x29: 0000ffffdb7a2f40 x28: 0000000000000000 [19321510.826710] x27: 0000000000000000 x26: 0000000000000000 [19321510.826714] x25: 0000000000000000 x24: 0000000000000000 [19321510.826718] x23: 0000000000000000 x22: 0000000000000000 [19321510.826723] x21: 0000ffffdb7a3068 x20: 000000003284d0e0 [19321510.826727] x19: 000000003284d0e0 x18: 0000000000000000 [19321510.826732] x17: 0000ffff7c1aab38 x16: 0000000000fe8148 [19321510.826736] x15: 003b9aca00000000 x14: 0000000000000000 [19321510.826741] x13: 0000000000000000 x12: 0000ffff7c041258 [19321510.826745] x11: 00000000004001c8 x10: 0000ffff7c041218 [19321510.826750] x9 : 0000000000000000 x8 : 0000ffffdb7a3068 [19321510.826754] x7 : 7f7f7f7fff7fff7f x6 : 0000000000000004 [19321510.826759] x5 : 00000000009c9e5c x4 : 0000000000fe8d28 [19321510.826763] x3 : 37fae0bca59bd300 x2 : 37fae0bca59bd300 [19321510.826767] x1 : 0000ffffdb7a2f80 x0 : 000000003284d0e0
Updated by Sebastian Wagner almost 5 years ago
Another one:
162/172 Test #169: unittest_seastar_messenger ..............***Exception: SegFault 18.11 sec WARNING: debug mode. Not for benchmarking or production WARN 2019-05-21 09:42:22,426 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,441 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,441 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,441 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,442 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,443 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,443 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,445 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,445 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,445 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,445 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,445 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,454 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,453 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,469 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:22,469 seastar - Seastar compiled with default allocator, heap profiler not supported WARN 2019-05-21 09:42:23,405 [shard 0] seastar - Unable to set SCHED_FIFO scheduling policy for timer thread; latency impact possible. Try adding CAP_SYS_NICE WARN 2019-05-21 09:42:23,692 [shard 0] seastar - membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED) is not available, reactor will not sleep when idle. Upgrade to Linux 4.14 or later INFO 2019-05-21 09:42:23,715 [shard 0] ms - test_echo(rounds=512, keepalive_ratio=0.1, v2=0): INFO 2019-05-21 09:42:23,819 [shard 0] ms - listening on 127.0.0.1:9010 INFO 2019-05-21 09:42:23,820 [shard 0] ms - listening on 127.0.0.1:9011 INFO 2019-05-21 09:42:23,971 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@59330 >> v1:127.0.0.1:9010/1]: connected to v1:127.0.0.1:9010/1 INFO 2019-05-21 09:42:23,974 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@62450 >> v1:127.0.0.1:9011/2]: connected to v1:127.0.0.1:9011/2 INFO 2019-05-21 09:42:23,984 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@59618 >> v1:127.0.0.1:9010/1]: connected to v1:127.0.0.1:9010/1 INFO 2019-05-21 09:42:24,010 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@55202 >> v1:127.0.0.1:9011/2]: connected to v1:127.0.0.1:9011/2 INFO 2019-05-21 09:42:25,950 [shard 0] ms - [osd.3(client2) v1:127.0.0.1:0/4@55202 >> v1:127.0.0.1:9011/2]: finished sending 512 pings with 59 keepalives INFO 2019-05-21 09:42:26,003 [shard 0] ms - [osd.2(client1) v1:127.0.0.1:0/3@59618 >> v1:127.0.0.1:9010/1]: finished sending 512 pings with 62 keepalives INFO 2019-05-21 09:42:29,077 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@59330 >> v1:127.0.0.1:9010/1]: finished sending 512 pings with 51 keepalives INFO 2019-05-21 09:42:29,220 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@62450 >> v1:127.0.0.1:9011/2]: finished sending 512 pings with 52 keepalives INFO 2019-05-21 09:42:36,687 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@59330 >> v1:127.0.0.1:9010/1]: finished receiving 512 pongs INFO 2019-05-21 09:42:36,690 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@59330 >> v1:127.0.0.1:9010/1]: handshake 0.104079, pingpong 12.7155 INFO 2019-05-21 09:42:36,693 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@59618 >> v1:127.0.0.1:9010/1]: finished receiving 512 pongs INFO 2019-05-21 09:42:36,694 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@59618 >> v1:127.0.0.1:9010/1]: handshake 0.116535, pingpong 12.7091 INFO 2019-05-21 09:42:36,716 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@62450 >> v1:127.0.0.1:9011/2]: finished receiving 512 pongs INFO 2019-05-21 09:42:36,720 [shard 2] ms - [osd.2(client1) v1:127.0.0.1:0/3@62450 >> v1:127.0.0.1:9011/2]: handshake 0.107043, pingpong 12.7415 INFO 2019-05-21 09:42:36,727 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@55202 >> v1:127.0.0.1:9011/2]: finished receiving 512 pongs INFO 2019-05-21 09:42:36,728 [shard 2] ms - [osd.3(client2) v1:127.0.0.1:0/4@55202 >> v1:127.0.0.1:9011/2]: handshake 0.142769, pingpong 12.7171 ISegmentation fault on shard 2. Backtrace: 0x0000000001e4aa6b 0x0000000001e3ff6f 0x0000000001db90bf 0x0000000001db9117 0x0000000001dd6fef 0x0000000001dea667 0x0000000001dea6cf 0x000000000000050f 0x0000ffffb2395bb7
https://jenkins.ceph.com/job/ceph-pull-requests-arm64/267/console
Updated by Kefu Chai almost 5 years ago
- Status changed from New to Resolved
- Assignee deleted (
Kefu Chai) - Pull request ID set to 28362
Actions