Project

General

Profile

Actions

Bug #50530

closed

pacific: client: abort after MDS blocklist

Added by Patrick Donnelly almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, libcephfs
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2021-04-23T04:04:42.325 INFO:tasks.ceph.mds.a.smithi092.stderr:2021-04-23T04:04:42.323+0000 7f649a4dd700 -1 MDSIOContextBase: blocklisted!  Restarting...
2021-04-23T04:04:42.341 INFO:tasks.ceph.mds.a.smithi092.stderr:  -229> 2021-04-23T04:04:42.322+0000 7f649a4dd700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown
2021-04-23T04:04:42.341 INFO:tasks.ceph.mds.a.smithi092.stderr:  -228> 2021-04-23T04:04:42.322+0000 7f649a4dd700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown
2021-04-23T04:04:42.342 INFO:tasks.ceph.mds.a.smithi092.stderr:    -2> 2021-04-23T04:04:42.323+0000 7f649a4dd700 -1 MDSIOContextBase: blocklisted!  Restarting...
2021-04-23T04:04:42.361 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at
2021-04-23T04:04:42.539 INFO:tasks.ceph.mon.a.smithi092.stderr:2021-04-23T04:04:42.537+0000 7f1f18b2c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN)
2021-04-23T04:04:42.545 INFO:tasks.ceph.mgr.y.smithi092.stderr:terminate called after throwing an instance of 'std::out_of_range'
2021-04-23T04:04:42.546 INFO:tasks.ceph.mgr.y.smithi092.stderr:  what():  map::at
2021-04-23T04:04:42.546 INFO:tasks.ceph.mgr.y.smithi092.stderr:*** Caught signal (Aborted) **
2021-04-23T04:04:42.546 INFO:tasks.ceph.mgr.y.smithi092.stderr: in thread 7f9eebc79700 thread_name:ms_dispatch
2021-04-23T04:04:42.546 INFO:tasks.ceph.mgr.y.smithi092.stderr: ceph version 16.2.1-116-g15a93ae9 (15a93ae92777636c0a2152882baa34390456e311) pacific (stable)
2021-04-23T04:04:42.546 INFO:tasks.ceph.mgr.y.smithi092.stderr: 1: /lib64/libpthread.so.0(+0x12b20) [0x7f9f5c2b4b20]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 2: gsignal()
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 3: abort()
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 4: /lib64/libstdc++.so.6(+0x9009b) [0x7f9f5b6bb09b]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 5: /lib64/libstdc++.so.6(+0x9653c) [0x7f9f5b6c153c]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 6: /lib64/libstdc++.so.6(+0x96597) [0x7f9f5b6c1597]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 7: /lib64/libstdc++.so.6(+0x967f8) [0x7f9f5b6c17f8]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 8: /lib64/libstdc++.so.6(+0x92045) [0x7f9f5b6bd045]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 9: /lib64/libcephfs.so.2(+0xab968) [0x7f9f4adc5968]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 10: (DispatchQueue::entry()+0x4f5) [0x7f9f5d903675]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9f5d9b40e1]
2021-04-23T04:04:42.547 INFO:tasks.ceph.mgr.y.smithi092.stderr: 12: /lib64/libpthread.so.0(+0x814a) [0x7f9f5c2aa14a]
2021-04-23T04:04:42.548 INFO:tasks.ceph.mgr.y.smithi092.stderr: 13: clone()

From: /ceph/teuthology-archive/yuriw-2021-04-22_16:46:11-fs-wip-yuri5-testing-2021-04-20-0819-pacific-distro-basic-smithi/6064805/teuthology.log


Related issues 1 (0 open1 closed)

Copied to CephFS - Backport #51324: pacific: pacific: client: abort after MDS blocklistResolvedPatrick DonnellyActions
Actions #1

Updated by Patrick Donnelly almost 3 years ago

  • Assignee set to Venky Shankar
Actions #2

Updated by Venky Shankar almost 3 years ago

  • Status changed from New to In Progress
Actions #3

Updated by Venky Shankar almost 3 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 41875
Actions #4

Updated by Patrick Donnelly almost 3 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #5

Updated by Backport Bot almost 3 years ago

  • Copied to Backport #51324: pacific: pacific: client: abort after MDS blocklist added
Actions #6

Updated by Loïc Dachary almost 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF