Documentation #62837: Add support for read_from_replica=localize for cephfs similar to krbd - Linux kernel client - Ceph

Actions

Copy link

Documentation #62837

open

Add support for read_from_replica=localize for cephfs similar to krbd

Added by Rakshith R 8 months ago. Updated 2 months ago.

Status:

In Progress

Priority:

Normal

Assignee:

Xiubo Li

Category:

Target version:

% Done:

Tags:

Backport:

Reviewed:

Affected Versions:

Description

KRBD supports serving reads from the OSD nearest to the client.
Refer to krbd-options¹ read_from_replica & crush_location.
After this feature was implemented in CephCSI² and Rook³, improvements in iops(increased by ~25%) and latency(decreased by ~90%) were seen on performance tests(Internal Rook deployed Ceph Cluster).
(memory usage of OSDs increased too as a side affect)

It'll be great to have the support for similar feature in CephFS.

references:
[1] https://docs.ceph.com/en/latest/man/8/rbd/#kernel-rbd-krbd-options
[2] https://github.com/ceph/ceph-csi/blob/907c4f38f6d68bd9701ebb7e12745ed919db17c7/docs/deploy-rbd.md#read-affinity-using-crush-locations-for-rbd-volumes
[3] https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Ceph-CSI/ceph-csi-drivers.md#enable-read-affinity-for-rbd-volumes

Actions

Copy link

Updated by Venky Shankar 8 months ago

Project changed from CephFS to Linux kernel client
Category deleted (~~Performance/Resource Usage~~)

`read_from_replica` is supported in libceph and I think you should be able to mount kceph by adding `read_from_replica=localize` as a mount option.

Actions

Copy link

Updated by Rakshith R 8 months ago

Venky Shankar wrote:

`read_from_replica` is supported in libceph and I think you should be able to mount kceph by adding `read_from_replica=localize` as a mount option.

Thanks for the quick reply.

Then the following mount command will work exactly how rbd map works ?
```
mount -t ceph -o mds_namespace=myfs,name=admin,secret=$my_secret,read_from_replica=localize,crush_location="host:compute1|region:north" $mon_endpoints:/ /tmp/registry
```

Actions

Copy link

Updated by Venky Shankar 8 months ago

Rakshith R wrote:

Venky Shankar wrote:

`read_from_replica` is supported in libceph and I think you should be able to mount kceph by adding `read_from_replica=localize` as a mount option.

Thanks for the quick reply.

Then the following mount command will work exactly how rbd map works ?
```
mount -t ceph -o mds_namespace=myfs,name=admin,secret=$my_secret,read_from_replica=localize,crush_location="host:compute1|region:north" $mon_endpoints:/ /tmp/registry
```

Yes. I think that should work.

Actions

Copy link

Updated by Rakshith R 6 months ago

Thanks,

Can we use this tracker to implement changes in cephfs mount options documentation ?
This way more users can discover this and benefit from it.

Actions

Copy link

Updated by Venky Shankar 6 months ago

Tracker changed from Feature to Documentation
Assignee set to Xiubo Li

Rakshith R wrote:

Thanks,

Can we use this tracker to implement changes in cephfs mount options documentation ?
This way more users can discover this and benefit from it.

Sure. Changing the tracker to Documentation - will get the doc bits up for review.

Xiubo, let's start adding some details regarding this in cephfs docs :)

Actions

Copy link