Project

General

Profile

Support #16526

cephfs client side quotas - nfs-ganesha

Added by sean redmond over 7 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Low
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Tags:
Reviewed:
Affected Versions:
Component(FS):
Client, Ganesha FSAL
Labels (FS):
quotas
Pull request ID:

Description

I am not sure If this is best logged on the nfs-ganesha project or here.

Ceph quotas are configured using virtual extended attributes:

ceph.quota.max_files – file limit
ceph.quota.max_bytes – byte limit

If the attributes appear on a directory inode that means a quota is configured there. If they are not present then no quota is set on that directory (although one may still be configured on a parent directory).

It seems that the ceph FSAL does not use this attribute and in turn quotas are ignored.

History

#1 Updated by Greg Farnum over 7 years ago

  • Project changed from Ceph to CephFS

How are you evaluating that the quotas are ignored? There isn't any integration, certainly, but the Ceph client library is responsible for enforcing quotas, and if they're present it will do so regardless of what Ganesha does.

I suppose it could run into trouble if Ganesha is caching, or make the server unhappy with weird return codes...?

#2 Updated by Greg Farnum over 7 years ago

  • Category set to 97
  • Component(FS) Client, Ganesha FSAL added

#3 Updated by sean redmond over 7 years ago

For this test I was using the below versions:

ceph version 10.2.1 (3a66dd4f30852819c1bdaa8ec23c795d4ad77269)
nfs-ganesha Starting: Version 2.1.1-pre

I have set a quota on /liam/ as shown below:

root@ceph-ganesha-1:/nfs# getfattr -n ceph.quota.max_bytes liam
  1. file: liam
    ceph.quota.max_bytes="100000000"

root@ceph-ganesha-1:/nfs# mount | grep nfs
ceph-fuse on /nfs type fuse.ceph-fuse (rw,nosuid,nodev,allow_other,default_permissions)

I have used cephfuse here just for the purpose of setting the ceph.quota.max_byte attribute.

I then use the below 'ganesha.conf' config to export this folder to a remote nfs client:

'EXPORT {
Export_ID = 1;
Path = "/liam";
Pseudo = "/liam";
NFS_Protocols = "4";
Transport_Protocols = TCP;
SecType = "sys";

FSAL {
Name = CEPH;
}
CLIENT {
Clients = x.x.x.x;
Access_Type = RW;
Squash = None;
}

}
'
The export works as expected, But the quota is never enforced, I can keep on writing new files taking up 100's GB of disk space.

As the Ceph client library is responsible for enforcing quotas maybe this is missing from the ganesha FSAL, even with debug logging enabled I don't see any reference to quotas at all.

#4 Updated by Greg Farnum over 7 years ago

  • Tracker changed from Feature to Support
  • Status changed from New to 4

Oh, we just recently flipped the bit so quotas are enforced by default. This should work if you set "client quota = true" in the ceph.conf (client section) used by Ganesha.

#5 Updated by Ramana Raja about 7 years ago

In a Ganesha (V2.5-dev-6) and Ceph (latest Jewel) setup, I set `client quota = true` in the client section of the ceph.conf (as suggested by Greg) used by Ganesha, and could see that quotas were enforced on the NFS share. Maybe this issue can be closed?

The client_quota option is enabled by default in Kraken (https://github.com/ceph/ceph/commit/48acd4b35c860589d43e7cce7a80b5a023fd9f21).

#6 Updated by John Spray about 7 years ago

  • Status changed from 4 to Resolved

Yep, I think so.

#7 Updated by Patrick Donnelly about 5 years ago

  • Category deleted (97)
  • Labels (FS) quotas added

Also available in: Atom PDF