Ceph » CephFS

Patrick Donnelly wrote:

The _deleting directory can often get sudden large volumes to recursively unlink. Rank 0 is not an ideal default target for this extra workload. We should just select rank 1 by default but introduce a configuration to alter that setting.

If max_mds==1, of course, the _deleting directory still stays on rank 0.

As an add-on to this, add a file to the directory which persists (i.e. not deleted by the async deleter threads) so that the directory is not exported back to rank 0 whenever it is empty.

Is that how the balancer works? I thought the dummy entry is required for the balancer to kick exporting without which the balancer would ignore the pin. I didn't know that the balancer would re-export to rank-0 (from rank-N) if a directory becomes empty irrespective of the pin.

Also, I think there is a catch to this feature. Commit aae7a70ed2cf9c32684cfdaf701778a05f229e09 introduces per subvolume trash directory with the /volumes/_deleting/ having a symlink entry to point to the deleted subvolume. The reason for this was that the subvolume is marked with ceph.dir.subvolume xattr which disables nested snapshots and renames across subvolumes.

Venky Shankar wrote:

I didn't know that the balancer would re-export to rank-0 (from rank-N) if a directory becomes empty irrespective of the pin.

Yes, it will. This is part of the merging subtree logic.

Venky Shankar wrote:

Also, I think there is a catch to this feature. Commit aae7a70ed2cf9c32684cfdaf701778a05f229e09 introduces per subvolume trash directory with the /volumes/_deleting/ having a symlink entry to point to the deleted subvolume. The reason for this was that the subvolume is marked with ceph.dir.subvolume xattr which disables nested snapshots and renames across subvolumes.

Ah, good point. In that case, this feature may be undesirable as this will work poorly with distributed ephemeral pins on the subvolume group.