Bug #51357: osd: sent kickoff request to MDS and then stuck for 15 minutes until MDS crash - CephFS - Ceph

Actions

Copy link

Bug #51357

closed

osd: sent kickoff request to MDS and then stuck for 15 minutes until MDS crash

Added by Xiubo Li almost 3 years ago. Updated 12 months ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Category:

Target version:

Ceph - v17.0.0

% Done:

100%

Source:

Q/A

Tags:

backport_processed

Backport:

pacific,octopus

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

Labels (FS):

Pull request ID:

42011

Crash signature (v1):

Crash signature (v2):

Description

    -3> 2021-06-16T23:37:26.998+0000 7fe145b22700 10 MDSIOContextBase::complete: 23C_IO_MDC_TruncateFinish
    -2> 2021-06-16T23:37:26.998+0000 7fe145b22700 10 MDSContext::complete: 23C_IO_MDC_TruncateFinish
    -1> 2021-06-16T23:37:26.999+0000 7fe145b22700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-5307-g06fb5cf0/rpm/el8/BUILD/ceph-17.0.0-5307-g06fb5cf0/src/mds/MDCache.cc: In function 'virtual void C_IO_MDC_TruncateFinish::finish(int)' thread 7fe145b22700 time 2021-06-16T23:37:26.999873+0000
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-5307-g06fb5cf0/rpm/el8/BUILD/ceph-17.0.0-5307-g06fb5cf0/src/mds/MDCache.cc: 6451: FAILED ceph_assert(r == 0 || r == -2)

 ceph version 17.0.0-5307-g06fb5cf0 (06fb5cf0031e099ece537a86a27543dc4010ce0c) quincy (dev)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x7fe15458400c]
 2: /usr/lib64/ceph/libceph-common.so.2(+0x27d214) [0x7fe154584214]
 3: ceph-mds(+0x3693a1) [0x5632175693a1]
 4: (MDSContext::complete(int)+0x56) [0x5632176c79a6]
 5: (MDSIOContextBase::complete(int)+0xa3) [0x5632176c7cd3]
 6: (Finisher::finisher_thread_entry()+0x18c) [0x7fe154627a7c]
 7: (Thread::_entry_func(void*)+0xd) [0x7fe15467ad2d]
 8: /lib64/libpthread.so.0(+0x814a) [0x7fe15331f14a]
 9: clone()

From: /ceph/teuthology-archive/pdonnell-2021-06-16_21:26:55-fs-wip-pdonnell-testing-20210616.191804-distro-basic-smithi/6175669/remote/smithi025/log/ceph-mds.e.log.gz

Looks like #44295 but that was during shutdown. This MDS is active.

Related issues 3 (0 open — 3 closed)