https://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2017-12-15T20:16:00ZCeph CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1038292017-12-15T20:16:00ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>4</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Low</i></li><li><strong>Target version</strong> set to <i>v13.0.0</i></li><li><strong>Component(FS)</strong> <i>Client</i> added</li></ul><p>What's the goal? Prevent the situation where the client has ~1M caps for an indefinite period like what we saw on the mailing list?</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1038332017-12-15T20:26:05ZPatrick Donnellypdonnell@redhat.com
<ul></ul><p>Ah, the problem is recovery of the MDS takes too long. (from follow-up posts to "[ceph-users] cephfs mds millions of caps")</p>
<p>I'd like to see measurements to see how long it takes the MDS to load metadata during recovery. Is there a way we could make that faster?</p>
<blockquote>
<p>tracking the rate we add new cap to the client</p>
</blockquote>
<p>Do we really care about the rate we add caps? If the client is bumping into its cache limit then it will begin trimming. We should only care if the caps are old and unused?</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1038392017-12-16T00:01:27ZZheng Yanukernel@gmail.com
<ul></ul><p>no about recovery time, clients already trim their cache aggressively when mds recovers.</p>
<p>Idle client holds so many caps is wasteful, because it increase the chance that mds trim other relatively hot objects from the cache.</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1039372017-12-19T23:45:29ZPatrick Donnellypdonnell@redhat.com
<ul></ul><p>Zheng Yan wrote:</p>
<blockquote>
<p>Idle client holds so many caps is wasteful, because it increase the chance that mds trim other relatively hot objects from the cache.</p>
</blockquote>
<p>Okay, that makes sense. I think the decay counter idea is good; let's do it.</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1044052018-01-05T18:46:54ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-3 priority-4 priority-default closed" href="/issues/21156">Feature #21156</a>: mds: speed up recovery with many open inodes</i> added</li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1129572018-05-09T21:14:54ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Status</strong> changed from <i>4</i> to <i>New</i></li><li><strong>Assignee</strong> set to <i>Rishabh Dave</i></li><li><strong>Priority</strong> changed from <i>Low</i> to <i>High</i></li><li><strong>Target version</strong> changed from <i>v13.0.0</i> to <i>v14.0.0</i></li><li><strong>Backport</strong> set to <i>mimic,luminous</i></li><li><strong>Component(FS)</strong> <i>MDS, kceph</i> added</li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1129772018-05-10T11:25:20ZWebert Limawebert.lima@mav.com.br
<ul></ul><p>Glad to see this :)</p>
<pre><code>- Backport set to mimic,luminous</code></pre>
<p>Thanks.</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1146472018-06-05T09:14:33ZRishabh Dave
<ul></ul><p>Can I get few implementation specific details to get started working on this issue?</p>
<p>And for clarity on my side, we would trim caps when the count reaches zero (and the counter reaches zero when there are no ops on the session for a certain while i.e. it is idle), right?</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1167252018-07-12T02:32:10ZPatrick Donnellypdonnell@redhat.com
<ul></ul><p>Rishabh Dave wrote:</p>
<blockquote>
<p>Can I get few implementation specific details to get started working on this issue?</p>
</blockquote>
<p>Copying from your email:</p>
<blockquote>
<ul>
<li>The issue description talks about "decay counter". I haven't found<br />to a source to verify what I understand by the term, but in my<br />understanding we'll increase the count with the every cap that the<br />client acquires and then if there are no ops for a certain period of<br />time, the count must naturally decrease until either there are no more<br />caps or there an op is conducted on the filesystem. Please correct me<br />if I am wrong here.</li>
</ul>
</blockquote>
<p>DecayCounter is a data structure used in the MDS in various places to track popularity of metadata (for the purposes of subtree migration).</p>
<blockquote>
<ul>
<li>The issue description says "tracking the rate we add new cap to<br />client". What is the purpose behind knowing the rate? In my<br />understanding, idle clients must trim caps irrespectively. Is it to<br />vary the timeout for each session accordingly?</li>
</ul>
</blockquote>
<p>I'm not sure having a DecayCounter for each session is quite right. We'd probably want a DecayCounter per-cap and trim caps that haven't been used in a long time (like 1 week?).</p>
<blockquote>
<ul>
<li>Should the code to implement this feature be implemented with MDS<br />(idle sessions are request to trim caps) or with client (sessions<br />trims the caps after the certain period of time)?</li>
</ul>
</blockquote>
<p>In the client. Only the client knows how often it uses its capabilities (i.e. how often some file I/O relies on a capability).</p>
<blockquote>
<ul>
<li>Currently, on the MDS side we already trim caps for idle sessions<br />after `session_timeout`[2]. So, how is this feature be different from<br />that?</li>
</ul>
</blockquote>
<p>Idle sessions are not the same as idle clients in this context. An idle client would be a client that hasn't done any significant file I/O recently (recently to-be-defined). Whereas an idle session is a concept in the MDS where the client has not spoken to the MDS in session_timeout seconds (e.g. hasn't renewed caps as required).</p> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1187022018-08-08T13:41:42ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Assignee</strong> deleted (<del><i>Rishabh Dave</i></del>)</li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1242892018-11-07T22:48:21ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>Assignee</strong> set to <i>Patrick Donnelly</i></li><li><strong>Priority</strong> changed from <i>High</i> to <i>Urgent</i></li><li><strong>Source</strong> set to <i>Development</i></li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1309552019-03-07T23:21:17ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Target version</strong> changed from <i>v14.0.0</i> to <i>v15.0.0</i></li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1394132019-06-22T02:46:39ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Start date</strong> deleted (<del><i>12/15/2017</i></del>)</li><li><strong>Backport</strong> changed from <i>mimic,luminous</i> to <i>nautilus</i></li><li><strong>Pull request ID</strong> set to <i>28702</i></li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1460982019-09-13T15:35:42ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Pending Backport</i></li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1461492019-09-16T00:55:10ZPatrick Donnellypdonnell@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed" href="/issues/41835">Bug #41835</a>: mds: cache drop command does not drive cap recall</i> added</li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1462012019-09-16T07:21:51ZNathan Cutlerncutler@suse.cz
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-9 status-3 priority-4 priority-default closed" href="/issues/41865">Backport #41865</a>: nautilus: mds: ask idle client to trim more caps</i> added</li></ul> CephFS - Feature #22446: mds: ask idle client to trim more capshttps://tracker.ceph.com/issues/22446?journal_id=1555972020-01-13T12:31:13ZNathan Cutlerncutler@suse.cz
<ul><li><strong>Status</strong> changed from <i>Pending Backport</i> to <i>Resolved</i></li></ul><p>While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".</p>