Bug #37726
closedmds: high debug logging with many subtrees is slow
0%
Description
In various places the MDS prints subtrees to the debug log. We should truncate the list if the number of subtrees is large.
Updated by Patrick Donnelly over 5 years ago
- Assignee changed from Patrick Donnelly to Rishabh Dave
Updated by Rishabh Dave over 5 years ago
I created 100 directories and wrote 10 files in each directory after pinning all the directories on a MDS. If I reproduced it correctly this must be the is the huge subtree you were talking about - https://paste.fedoraproject.org/paste/3E5EtAAEcdqOHovpxMWTlw ?
What should the threshold be for deciding that the number of subtrees is huge? And since we would be truncating the list, do we have a preference on which subtrees we would like to print? Also, at some other log level do we want to print all the subtrees?
Updated by Patrick Donnelly over 5 years ago
- Status changed from New to In Progress
Rishabh Dave wrote:
I created 100 directories and wrote 10 files in each directory after pinning all the directories on a MDS. If I reproduced it correctly this must be the is the huge subtree you were talking about - https://paste.fedoraproject.org/paste/3E5EtAAEcdqOHovpxMWTlw ?
Yes, that's it!
What should the threshold be for deciding that the number of subtrees is huge? And since we would be truncating the list, do we have a preference on which subtrees we would like to print? Also, at some other log level do we want to print all the subtrees?
If the list is large, I think the default should be printing the number of subtrees and maybe some other metadata in a single message. Maybe if debugging is >=25 then we should always print the full list.
I think MDBalancer::handle_heartbeat could be the place that prints the full list. That should happen every few seconds.
Updated by Patrick Donnelly about 5 years ago
- Target version changed from v14.0.0 to v15.0.0
Updated by Patrick Donnelly about 5 years ago
- Category set to Performance/Resource Usage
- Target version changed from v15.0.0 to v14.0.0
Updated by Patrick Donnelly about 5 years ago
- Target version changed from v14.0.0 to v15.0.0
- Backport changed from mimic,luminous to nautilus,mimic,luminous
- Pull request ID set to 26056
Updated by Patrick Donnelly about 5 years ago
- Status changed from In Progress to Pending Backport
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #38875: mimic: mds: high debug logging with many subtrees is slow added
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #38876: nautilus: mds: high debug logging with many subtrees is slow added
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #38877: luminous: mds: high debug logging with many subtrees is slow added
Updated by Nathan Cutler almost 5 years ago
- Status changed from Pending Backport to Resolved
Updated by Nathan Cutler almost 5 years ago
- Status changed from Resolved to Pending Backport
mimic backport is still open
Updated by Nathan Cutler over 4 years ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".