Project

General

Profile

Actions

Bug #58362

closed

unittest-seastar-messenger-thrash error

Added by Xuehan Xu over 1 year ago. Updated 11 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

INFO 2022-12-26 07:23:11,895 seastar - Reactor backend: linux-aio
WARN 2022-12-26 07:23:11,896 [shard 0] seastar - Creation of perf_event based stall detector failed, falling back to posix timer: std::system_error (error system:13, perf_event_open() failed: Permission denied)
INFO 2022-12-26 07:23:11,900 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000
INFO 2022-12-26 07:23:11,900 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0
INFO 2022-12-26 07:23:11,900 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647
INFO 2022-12-26 07:23:11,900 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000
Reactor stalled for 140 ms on shard 0. Backtrace: 0x552f43a 0x86619e4 0x8660ebc 0x847ee43 0x847988f 0x8479337 0x8479d11 0x84802ca 0x4251f 0x5575bcf 0x5575b0d 0x5f9fa26 0x5fb7ead 0x5fb702b 0x5fb647e 0x5f4658c 0x5f070e1 0x7946bd1 0x5650382 0x564ff68 0x564fc27 0x564f90b 0x564f7aa 0x564f5bf 0x564f29f 0x564f043 0x564ea87 0x564e7d9 0x564dfef 0x564d751 0x564d2f8 0x564ce59 0x564cba2 0x564c927 0x564ba47 0x88e7701 0x88e7396 0x88e7267 0x88e6cac 0x88e66e2 0x88e7905 0x88e68cf 0x88e6344 0x88e607f 0x56456ce 0x5616ca6 0x55b3ef3 0x5612be2 0x5612acf 0x561294f 0x561254f 0x82b3944 0x82b36f3 0x82b32ff 0x826a547 0x826a19c 0x826a04c 0x8269a6c 0x5b1c3d1 0x82bf4cf 0x82bf45f 0x82bf3ad 0x82bf1cb 0x5699b2b 0x56999af 0x56998ff 0x56997ef 0x56995e3 0x569952c 0x56992bc 0x56990b6 0x5698bca 0x84a194d 0x84ac86e 0x84b22cb 0x84afe98 0x8266c82 0x82640f6 0x55b57a0 0x29d8f 0x29e3f 0x54f3904
INFO 2022-12-26 07:23:12,211 [shard 0] test - test_stress():
Reactor stalled for 128 ms on shard 0. Backtrace: 0x552f43a 0x86619e4 0x8660ebc 0x847ee43 0x847988f 0x8479337 0x8479d11 0x84802ca 0x4251f 0x2a2795d86bb9 0x5575db3 0x5e6ac89 0x55b8c56 0x55b7c28 0x55b735d 0x55b6d03 0x55b6c8f 0x55b6bdd 0x55b6b4b 0x5699b2b 0x56999af 0x56998ff 0x56997ef 0x56995e3 0x569952c 0x56992bc 0x56990b6 0x5698bca 0x84a194d 0x84ac86e 0x84b22cb 0x84afe98 0x8266c82 0x82640f6 0x55b57a0 0x29d8f 0x29e3f 0x54f3904
Reactor stalled for 317 ms on shard 0. Backtrace: 0x552f43a 0x86619e4 0x8660ebc 0x847ee43 0x847988f 0x8479337 0x8479d11 0x84802ca 0x4251f 0x558f920 0x5591219 0x559aefa 0x559af7f 0x55992cb 0x54f6161 0x54f6b35 0x557732b 0x5e8dedc 0x5e602b0 0x5e5ff93 0x5e5ffcd 0x5e616b4 0x55b8c40 0x55b7c28 0x55b735d 0x55b6d03 0x55b6c8f 0x55b6bdd 0x55b6b4b 0x5699b2b 0x56999af 0x56998ff 0x56997ef 0x56995e3 0x569952c 0x56992bc 0x56990b6 0x5698bca 0x84a194d 0x84ac86e 0x84b22cb 0x84afe98 0x8266c82 0x82640f6 0x55b57a0 0x29d8f 0x29e3f 0x54f3904
INFO 2022-12-26 07:23:12,793 [shard 0] ms - [osd.2(server) v2:127.0.0.1:16801/2] try_bind: done
INFO 2022-12-26 07:23:12,794 [shard 0] ms - [osd.2(server) v2:127.0.0.1:16801/2] try_bind: done
INFO 2022-12-26 07:23:12,795 [shard 0] ms - [osd.3(server) v2:127.0.0.1:16802/3] was unable to bind. Trying again in 5 seconds
INFO 2022-12-26 07:23:17,812 [shard 0] ms - [osd.3(server) v2:127.0.0.1:16802/3] was unable to bind. Trying again in 5 seconds
INFO 2022-12-26 07:23:22,813 [shard 0] ms - [osd.3(server) v2:127.0.0.1:16802/3] was unable to bind. Trying again in 5 seconds
INFO 2022-12-26 07:23:27,815 [shard 0] ms - [osd.3(server) v2:127.0.0.1:16802/3] was unable to bind after 3 attempts: generic:98
ERROR 2022-12-26 07:23:27,815 [shard 0] test - operator() test_messenger_thrash(): there is another instance running at v2:127.0.0.1:16802/0
ERROR 2022-12-26 07:23:27,815 [shard 0] none - /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/crimson/test_messenger_thrash.cc:361 : In function 'auto (anonymous namespace)::SyntheticWorkload::init_server(const entity_name_t &, const std::string &, const uint64_t, const entity_addr_t &)::(anonymous class)::operator()(const std::error_code &) const', abort(%s)
abort() called
Aborting on shard 0.
Backtrace:
0x552f43a
0x86610c4
0x8660ce3
0x847edd9
0x84b3dd5
0x85bec69
0x85bee9e
0x85becea
/lib/x86_64-linux-gnu/libc.so.6+0x4251f
/lib/x86_64-linux-gnu/libc.so.6+0x96a7b
/lib/x86_64-linux-gnu/libc.so.6+0x42475
/lib/x86_64-linux-gnu/libc.so.6+0x287f2
0x792dc32
0x55cb45c
0x55cb074
0x55caf64
0x55cae54
0x55cad41
0x55cac7d
0x55cab2d
0x55ca95d
0x55ca6f7
0x55ca44d
0x55ca35d
0x55ca26d
0x55c93da
0x55c89b6
0x55c82c4
0x55c808b
0x55c7fa5
0x55c7d23
0x565f8e3
0x565f3bb
0x565edcc
0x565eb57
0x565e3aa
0x84a194d
0x84ac86e
0x84b22cb
0x84afe98
0x8266c82
0x82640f6
0x55b57a0
/lib/x86_64-linux-gnu/libc.so.6+0x29d8f
/lib/x86_64-linux-gnu/libc.so.6+0x29e3f
0x54f3904
Actions #1

Updated by Xuehan Xu over 1 year ago

  • Subject changed from messenger unit test error to unittest-seastar-messenger-thrash error
Actions #2

Updated by Yingxin Cheng over 1 year ago

From log: “there is another instance running at v2:127.0.0.1:16802/0”

That instance must be from the previous make check run. The environment was not cleared properly so the same port is still busy.

Actions #3

Updated by Matan Breizman 11 months ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF