Project

General

Profile

Feature #40986

cephfs qos: implement cephfs qos base on tokenbucket algorighm

Added by songbo wang 29 days ago. Updated 19 days ago.

Status:
Need Review
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Reviewed:
Affected Versions:
Component(FS):
Client, Common/Protocol, MDS
Labels (FS):
Pull request ID:

Description

The basic idea is as follows:

Set QoS info as one of the dir's xattrs;
All clients that can access the same dirs with the same QoS setting.
Follow the Quota's config flow. when the MDS receives the QoS setting, it'll also broadcast the message to all clients.
We can change the limit online.
[support]:
limit && burst config

[usage]:
setfattr -n ceph.qos.limit.iops -v 200 /mnt/cephfs/testdirs/
setfattr -n ceph.qos.burst.read_bps -v 200 /mnt/cephfs/testdirs/
getfattr -n ceph.qos.limit.iops /mnt/cephfs/testdirs/
getfattr -n ceph.qos /mnt/cephfs/testdirs/

[problems]:
Because there is no queue in CephFS IO path, If the bps is lower than the request's block size, the whole Client will be blocked until it gets enough token.

History

#1 Updated by songbo wang 29 days ago

I think there are two kinds of design:
1. all clients use the same QoS setting, just as the implementation
in this PR. Maybe there are multiple mount points, if we limit the
total IO, the number of total mount points is also limited.
So in my implementation, the total IO & BPS is not limited.

2. all clients share a specific QoS setting. I think there are two
kinds of use cases in detail.
2.1 setting a total limit, all clients limited by the average:
total_limit/clients_num.
2.2 setting a total limit, the mds decide the client's limitation by
their historical IO&BPS.

#2 Updated by Greg Farnum 26 days ago

  • Project changed from Ceph to fs

#3 Updated by Patrick Donnelly 19 days ago

  • Status changed from New to Need Review
  • Assignee set to songbo wang
  • Target version set to v15.0.0
  • Start date deleted (07/26/2019)
  • Source set to Community (dev)
  • Component(FS) Client, Common/Protocol, MDS added

Also available in: Atom PDF