https://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2016-07-15T02:25:21ZCeph CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=748192016-07-15T02:25:21ZZheng Yanukernel@gmail.com
<ul></ul><p>what do you mean they are old? what does 'rados stat xxxx' show?</p> CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=748852016-07-18T13:48:41ZJohn Sprayjcspray@gmail.com
<ul><li><strong>Assignee</strong> set to <i>Greg Farnum</i></li></ul> CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=748882016-07-18T13:53:32ZJohn Sprayjcspray@gmail.com
<ul></ul><p>Plan is for greg to look into the TMAP2OMAP OSD code to look for what might have causd that.</p>
<p>Afterwards John+Doug will get into trying to clean up the cluster with our repair tools.</p> CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=749272016-07-18T20:19:39ZGreg Farnumgfarnum@redhat.com
<ul><li><strong>Subject</strong> changed from <i>sepia LRC lost directories (tmap2omap went bad?)</i> to <i>sepia LRC lost directories</i></li><li><strong>Category</strong> changed from <i>Correctness/Safety</i> to <i>fsck/damage handling</i></li><li><strong>Assignee</strong> changed from <i>Greg Farnum</i> to <i>John Spray</i></li></ul><p>Well, I checked the code again and the tmap2omap path looks appropriately durable.</p>
<p>I did notice one thing that helps explain it a little: we pass a "nullok" flag when invoking tmap2omap, which makes the operation succeed even if there is no data present in the object. This is required, since if the directory is already an omap, there is no tmap data. But it means a previously-broken object won't get detected during this upgrade. :(</p>
<p>Anyway, this cluster has been damaged in various ways in the past. I think these directories simply got broken in the depths of time and are only now being noticed (so, hurray damage detection!).</p>
<p>Assigning to John for him and Doug to clean up.</p> CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=751932016-07-22T14:22:03ZJohn Sprayjcspray@gmail.com
<ul></ul><p>(Mainly for my reference) etherpad from repairing is here <a class="external" href="http://etherpad.corp.redhat.com/efev9SA7rn">http://etherpad.corp.redhat.com/efev9SA7rn</a></p> CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=752462016-07-23T21:59:50ZJohn Sprayjcspray@gmail.com
<ul><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>High</i></li></ul><p>The offending dentries that point to damaged dirfrags have been removed (by removing the omap keys). The objects themselves are still in the system but not in a way that will make it unhappy.</p> CephFS - Bug #16691: sepia LRC lost directorieshttps://tracker.ceph.com/issues/16691?journal_id=829672016-12-13T14:21:49ZJohn Sprayjcspray@gmail.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li></ul>