Project

General

Profile

Actions
Copy link

Bug #47645

open

mgr: self-test fails on devicehealth module

Added by Sunny Kumar over 3 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
devicehealth module
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Steps to reproduce:

1. MDS=0 MGR=1 OSD=2 MON=1 ../src/vstart.sh -n -b --without-dashboard --debug
2. ./bin/ceph mgr module enable selftest
3. /bin/ceph mgr self-test module devicehealth

Output:

Error EPERM: Test failed: Remote method threw exception: Traceback (most recent call last):
File "/home/sunny/lab/ceph/src/pybind/mgr/devicehealth/module.py", line 214, in self_test
assert before != after
AssertionError

-------------
mgr.x.log
-------------
2020-09-24T14:25:21.939+0100 7f4950f5c700 0 [devicehealth ERROR root] Fail to parse JSON result from daemon mon.a ()
2020-09-24T14:25:21.939+0100 7f4950f5c700 1 -- 192.168.0.5:0/2012805582 --> [v2:192.168.0.5:6810/13776,v1:192.168.0.5:6811/13776] -- osd_op(unknown.0.0:2 1.0 1:9eeed0c4:::SAMSUNG_MZVLB256HAHQ-000L7_S41GNX3M542660:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e16) v8 -- 0x558e62c99400 con 0x558e62ba3180
2020-09-24T14:25:21.940+0100 7f49714db700 1 -- 192.168.0.5:0/2012805582 <== osd.1 v2:192.168.0.5:6810/13776 2 ==== osd_op_reply(2 SAMSUNG_MZVLB256HAHQ-000L7_S41GNX3M542660 [omap-get-vals] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 185+0+0 (secure 0 0 0) 0x558e62d02240 con 0x558e62ba3180
2020-09-24T14:25:21.941+0100 7f4950f5c700 20 mgr ~Gil Destroying new thread state 0x558e62a6d9e0
2020-09-24T14:25:21.941+0100 7f4950f5c700 -1 Remote method threw exception: Traceback (most recent call last):
File "/home/sunny/lab/ceph/src/pybind/mgr/devicehealth/module.py", line 214, in self_test
assert before != after
AssertionError

2020-09-24T14:25:21.941+0100 7f4950f5c700 20 mgr ~Gil Destroying new thread state 0x558e62974d80
2020-09-24T14:25:21.941+0100 7f4950f5c700 -1 mgr.server reply reply (1) Operation not permitted Test failed: Remote method threw exception: Traceback (most recent call last):
File "/home/sunny/lab/ceph/src/pybind/mgr/devicehealth/module.py", line 214, in self_test
assert before != after
AssertionError

2020-09-24T14:25:21.941+0100 7f4950f5c700 1 -- [v2:192.168.0.5:6800/14559,v1:192.168.0.5:6801/14559] --> 192.168.0.5:0/314104114 -- mgr_command_reply(tid 0: -1 Test failed: Remote method threw exception: Traceback (most recent call last):
File "/home/sunny/lab/ceph/src/pybind/mgr/devicehealth/module.py", line 214, in self_test
assert before != after
AssertionError

Actions
Copy link
 #1

Updated by Sunny Kumar over 3 years ago

  • Category set to ceph-mgr
Actions
Copy link
 #2

Updated by Neha Ojha over 3 years ago

  • Category changed from ceph-mgr to devicehealth module
Actions
Copy link
 #3

Updated by Sunny Kumar over 3 years ago

mon.a.log
---------

2020-09-28T15:16:18.996+0100 7fd94885d700 0 log_channel(audit) log [INF] : from='client.? 192.168.0.5:0/2728662772' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "selftest"}]': finished
2020-09-28T15:16:18.996+0100 7fd94885d700 1 -- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] --> [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] -- log(1 entries from seq 73 at 2020-09-28T15:16:18.997887+0100) v1 -- 0x559954f49340 con 0x559953aee480
2020-09-28T15:16:18.996+0100 7fd94885d700 2 mon.a@0(leader) e1 send_reply 0x5599547d5440 0x559954dced00 mon_command_ack([{"prefix": "mgr module enable", "module": "selftest"}]=0 v6) v1
2020-09-28T15:16:18.996+0100 7fd94885d700 1 -- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] --> 192.168.0.5:0/2728662772 -- mon_command_ack([{"prefix": "mgr module enable", "module": "selftest"}]=0 v6) v1 -- 0x559954dced00 con 0x559954c15b00
2020-09-28T15:16:18.996+0100 7fd94a060700 1 -- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] <== mon.0 v2:192.168.0.5:40944/0 0 ==== log(1 entries from seq 73 at 2020-09-28T15:16:18.997887+0100) v1 ==== 0+0+0 (unknown 0 0 0) 0x559954f49340 con 0x559953aee480
2020-09-28T15:16:18.996+0100 7fd94885d700 0 log_channel(cluster) log [DBG] : mgrmap e6: x(active, since 10m)
2020-09-28T15:16:18.996+0100 7fd94885d700 1 -- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] --> [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] -- log(1 entries from seq 74 at 2020-09-28T15:16:18.998055+0100) v1 -- 0x559954f49dc0 con 0x559953aee480
2020-09-28T15:16:18.997+0100 7fd94885d700 20 mon.a@0(leader).paxos(paxos active c 252..762) finish_round waiting_for_readable
2020-09-28T15:16:18.997+0100 7fd94885d700 20 mon.a@0(leader).paxos(paxos active c 252..762) finish_round waiting_for_writeable
2020-09-28T15:16:18.997+0100 7fd94885d700 10 mon.a@0(leader).paxos(paxos active c 252..762) finish_round done w/ waiters, state active
2020-09-28T15:16:18.997+0100 7fd94885d700 10 mon.a@0(leader).paxos(paxos active c 252..762) propose_pending 763 29551 bytes
2020-09-28T15:16:18.997+0100 7fd94885d700 10 mon.a@0(leader).paxos(paxos updating c 252..762) begin for 763 29551 bytes
2020-09-28T15:16:19.000+0100 7fd94885d700 10 mon.a@0(leader).paxos(paxos updating c 252..762) commit_start 763
2020-09-28T15:16:19.000+0100 7fd94e068700 1 -- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] >> 192.168.0.5:0/2728662772 conn(0x559954c15b00 msgr2=0x559954f85900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20
2020-09-28T15:16:19.000+0100 7fd94a060700 20 mon.a@0(leader) e1 _ms_dispatch existing session 0x5599547ae900 for mon.0
2020-09-28T15:16:19.000+0100 7fd94a060700 20 mon.a@0(leader) e1 entity caps allow *
2020-09-28T15:16:19.000+0100 7fd94e068700 1 -- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] >> 192.168.0.5:0/2728662772 conn(0x559954c15b00 msgr2=0x559954f85900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed
2020-09-28T15:16:19.000+0100 7fd94e068700 1 --2- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] >> 192.168.0.5:0/2728662772 conn(0x559954c15b00 0x559954f85900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x559956615e00 tx=0x559954dd4720).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted)
2020-09-28T15:16:19.000+0100 7fd94a060700 10 mon.a@0(leader).paxosservice(logm 1..313) dispatch 0x559954f49340 log(1 entries from seq 73 at 2020-09-28T15:16:18.997887+0100) v1 from mon.0 v2:192.168.0.5:40944/0 con 0x559953aee480
2020-09-28T15:16:19.000+0100 7fd94e068700 1 --2- [v2:192.168.0.5:40944/0,v1:192.168.0.5:40945/0] >> 192.168.0.5:0/2728662772 conn(0x559954c15b00 0x559954f85900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x559956615e00 tx=0x559954dd4720).stop
2020-09-28T15:16:19.000+0100 7fd94a060700 5 mon.a@0(leader).paxos(paxos writing c 252..762) is_readable = 1 - now=2020-09-28T15:16:19.001681+0100 lease_expire=1970-01-01T01:00:00.000000+0100 has v0 lc 762
2020-09-28T15:16:19.000+0100 7fd94a060700 10 mon.a@0(leader).log v313 preprocess_query log(1 entries from seq 73 at 2020-09-28T15:16:18.997887+0100) v1 from mon.0 v2:192.168.0.5:40944/0
2020-09-28T15:16:19.000+0100 7fd94a060700 10 mon.a@0(leader).log v313 preprocess_log log(1 entries from seq 73 at 2020-09-28T15:16:18.997887+0100) v1 from mon.0

Actions
Copy link

Also available in: Atom PDF