Ceph

h1. Osd - tiering - reduce readwrite latencies on cache tier miss

h3. Summary

For a read, when the object is missing in cache tier, after promoting it from the base tier, the current code first replicates this object. Then the read request is served after the replication. We can serve the read request after the promotion without waiting for the completion of the replication to reduce latency.

For a write, when the object is missing in cache tier, after promoting it from the base tier, the current code replicates the object just promoted. And then updating the object with the data from client write. Another replication is needed for the client write. The whole write operation takes two replications. If we don’t replicate the object after the promotion, instead we replicate it after the write request has updated the data, we can save the time for one replication.

For a full object write, there is no need to promote the object data since it is going to be overwritten anyway. Only the object attributes and omap need to be promoted.

h3. Owners

* Zhiqiang Wang (intel)

* Jian Zhang (intel)

* Jiangang Duan (intel)

h3. Interested Parties

* Name (Affiliation)

* Name (Affiliation)

* Name

h3. Current Status

h3. Detailed Description

1) Read:

- In '_write_copy_chunk', based on the read request's offset and length, copy the corresponding promoted data to the 'outdata' of OSDOp. Mark the requested data is in the OSDOp after the copying.

- When the read request is unblocked and dequeued after the promotion, in do_op, if the requested data is already in the 'outdata' of OSDOp, don't need to get rw lock. Otherwise the read request is enqueued again to wait for the completion of the promotion replication.

- In 'do_osd_ops', if the requested data is already in the 'outdata' of OSDOp, do not need to read data from the backend objectstore.

2) Write:

- In '_write_copy_chunk', new a OpContext to store all the data promoted from base tier. Don't initiate repop to do replications in 'process_copy_chunk' and 'finish_promote".

- When the write request is unblocked and dequeued after the promotion, in do_op, use the OpContext in the previous step for the write, don't need to new another one.

- In 'do_osd_ops', update the OpContext with the data from client write, and submit this OpContext for replications.

3) Write full:

- In 'start_copy', set 'data_complete' of 'object_copy_cursor_t' in the 'cop' to true before promotion. So that when the base tier won't copy the data when handling the copy get request.

- The other code path is the same as in the previous write operation.

h3. Work items

h4. Coding tasks

# Task 1

# Task 2

# Task 3

h4. Build / release tasks

# Task 1

# Task 2

# Task 3

h4. Documentation tasks

# Task 1

# Task 2

# Task 3

h4. Deprecation tasks

# Task 1

# Task 2

# Task 3