Bug #14048: MDS heap stats standby threads - CephFS - Ceph

Actions

Copy link

Bug #14048

closed

MDS heap stats standby threads

Added by Vitaliy Dutchak over 8 years ago. Updated over 8 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

John Spray

Category:

Target version:

% Done:

Source:

other

Tags:

mds, heap stats

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v0.94.3, Ceph - v0.94.5, Ceph - v9.2.1

ceph-qa-suite:

Component(FS):

Labels (FS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Hello I am have a problem with heap stats (dump, start_profiler, stop_profiler) command:

When I run it - i get some results, but ceph-mds create 4 new threads and don't exit it, so if I call
ceph tell mds.a heap stats every 10 seconds after 5-6 hours I had 4000+ threads and cluster slow down.

in ceph-mds log i had a lot of:

2015-12-10 15:39:00.121346 7f6187fe6700 0 -- 10.10.10.11:6800/25374 >> 10.10.10.11:0/1766420499 pipe(0x7f61972f8000 sd=17 :6800 s=2 pgs=2 cs=1 l=0 c=0x7f6197194b00).fault, server, going to standby
2015-12-10 15:39:00.152101 7f6187de4700 0 -- 10.10.10.11:6800/25374 >> 10.10.10.11:0/1665626131 pipe(0x7f61972fd000 sd=18 :6800 s=2 pgs=2 cs=1 l=0 c=0x7f6197194dc0).fault, server, going to standby

I was try versions: 0.94.3, 0.94.5, 9.2 on all same picture.
I was try for other daemons (osd and mon) and it's work nice, but not for mds