cephfs qos: implement cephfs qos base on tokenbucket algorighm
The basic idea is as follows:
Set QoS info as one of the dir's xattrs;
All clients that can access the same dirs with the same QoS setting.
Follow the Quota's config flow. when the MDS receives the QoS setting, it'll also broadcast the message to all clients.
We can change the limit online.
limit && burst config
setfattr -n ceph.qos.limit.iops -v 200 /mnt/cephfs/testdirs/
setfattr -n ceph.qos.burst.read_bps -v 200 /mnt/cephfs/testdirs/
getfattr -n ceph.qos.limit.iops /mnt/cephfs/testdirs/
getfattr -n ceph.qos /mnt/cephfs/testdirs/
Because there is no queue in CephFS IO path, If the bps is lower than the request's block size, the whole Client will be blocked until it gets enough token.
#1 Updated by songbo wang 4 months ago
I think there are two kinds of design:
1. all clients use the same QoS setting, just as the implementation
in this PR. Maybe there are multiple mount points, if we limit the
total IO, the number of total mount points is also limited.
So in my implementation, the total IO & BPS is not limited.
2. all clients share a specific QoS setting. I think there are two
kinds of use cases in detail.
2.1 setting a total limit, all clients limited by the average:
2.2 setting a total limit, the mds decide the client's limitation by
their historical IO&BPS.