Ceph » CephFS

As per Venky, we need to keep in mind the following limitation with CephFS quotas:
"Quotas must be configured carefully when used with path-based mount restrictions. The client needs to have access to the directory inode on which quotas are configured in order to enforce them. If the client has restricted access to a specific path (e.g., /home/user) based on the MDS capability, and a quota is configured on an ancestor directory they do not have access to (e.g., /home), the client will not enforce it. When using path-based access restrictions be sure to configure the quota on the directory the client is restricted too (e.g., /home/user) or something nested beneath it."

In the case of manila, the OpenStack clients are expected to mount FS subvolumes using auth IDs with subvolume path based restrictions. This would mean that the clients wouldn't obey the subvolume group quota restrictions.

In the case of CSI, the subvolumes are mounted using auth IDs that don't have subvolume path based restrictions. See, https://github.com/ceph/ceph-csi/blob/devel/docs/capabilities.md#cephfs . So I think for the CSI use case the subvolume group quota restrictions would be honored.

Thanks, Ramana for the follow-up and validation of the use case.

Elaborating a bit more on the restriction mentioned by Ramana in the first comment.

If the quota is set on the subvolumegroup, in order for it get enforced for the data written to subvolumes with in that subvolume group, the mds caps plays the important role.
The 'user' which is accessing the subvolume (in csi use case, this could be 'node' which mounts the subvolume for read/write) should have atleast read mds caps for subvolumegroup
along with rw access to subvolume path.

There are two cases based on your use case.
1. If the user is restricted at subvolume level, it should have atleast read mds caps to others. Something like below

      mds allow r allow rw path=<path-to-subvolume>

2. If the user is restricted at subvolumegroup level, it should work by default as the caps would be something like below.

      mds allow rw path=<path-to-subvolumegroup>

Introducing subvolumegroup quotas hits this known issue [1]. The issue is hit while removing subvolume
under the subvolume group (whose quota is set) as removing subvolumes renames them out of the
subvolume group directory (quota root directory). We need to think about solving the issue for
this use case.

[1] https://tracker.ceph.com/issues/16884