Project

General

Profile

Feature #36253

cephfs: clients should send usage metadata to MDSs for administration/monitoring

Added by Patrick Donnelly over 5 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Performance/Resource Usage
Target version:
% Done:

0%

Source:
Development
Tags:
Backport:
Reviewed:
Affected Versions:
Component(FS):
Client, MDS, kceph
Labels (FS):
Pull request ID:

Description

In particular:

  • The capability "cache hits" by clients to provide introspection on the effectiveness of client capability caching. (Slightly different from #26996, this provides aggregate information for all caps from the client.)
  • OSDC cache hits.
  • OSDC readahead utilization (?); i.e. if the client reads ahead, does its workload actually use those buffers?
  • data write/read ops/bandwidth

In terms of implementation, this will be a new message sent to one of the ranks the client has a session with. It is probably not important which rank it sends the message to or that the same rank gets the sequential messages. Periodically, the ranks will forward these messages in a reduced form to rank 0. Rank 0, in a separate thread, can then aggregate all session metadata and send statistics to the mgr. This will probably include a snapshot of all sessions with various per-session statistics (#24285). (It's probably a good idea to send all of the sessions statistics to the mgr even if that is potentially ~1000 sessions. The user of `ceph fs top` may want to search for a particular session.)


Related issues

Related to CephFS - Feature #26996: cephfs: get capability cache hits by clients to provide introspection on effectiveness of client caching Resolved
Related to CephFS - Feature #24285: mgr: add module which displays current usage of file system (`fs top`) Resolved
Related to Linux kernel client - Feature #43215: kclient: send usage metrics to MDS Resolved
Related to CephFS - Bug #43251: mds: track client provided metric flags in session Resolved
Duplicated by CephFS - Feature #39982: cephfs client periodically report cache utilisation to MDS server Duplicate 05/20/2019

History

#1 Updated by Patrick Donnelly over 5 years ago

  • Related to Feature #26996: cephfs: get capability cache hits by clients to provide introspection on effectiveness of client caching added

#2 Updated by Patrick Donnelly over 5 years ago

  • Related to Feature #24285: mgr: add module which displays current usage of file system (`fs top`) added

#3 Updated by Patrick Donnelly over 5 years ago

  • Description updated (diff)

#4 Updated by Venky Shankar over 5 years ago

  • Status changed from New to In Progress
  • Assignee set to Venky Shankar

#5 Updated by Patrick Donnelly about 5 years ago

  • Target version changed from v14.0.0 to v15.0.0

#6 Updated by Patrick Donnelly almost 5 years ago

  • Duplicated by Feature #39982: cephfs client periodically report cache utilisation to MDS server added

#7 Updated by Venky Shankar over 4 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 26004

#8 Updated by Patrick Donnelly over 4 years ago

#9 Updated by Venky Shankar over 4 years ago

  • Related to Bug #43251: mds: track client provided metric flags in session added

#10 Updated by Patrick Donnelly about 4 years ago

  • Target version changed from v15.0.0 to v16.0.0

#11 Updated by Patrick Donnelly almost 4 years ago

  • Status changed from Fix Under Review to Resolved

\o/

Also available in: Atom PDF