Project

General

Profile

Bug #24173

ceph_volume_client: allow atomic update of RADOS objects

Added by Ramana Raja 7 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Manila
Target version:
Start date:
05/18/2018
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
mimic,luminous
Regression:
No
Severity:
5 - suggestion
Reviewed:
Affected Versions:
ceph-qa-suite:
fs
Component(FS):
VolumeClient
Labels (FS):
task(intern)
Pull request ID:

Description

The manila driver needs the ceph_volume_client to atomically update contents
of RADOS objects used to store ganesha's export counter and ganesha's
export RADOS URL index.


Related issues

Copied to fs - Backport #24862: luminous: ceph_volume_client: allow atomic update of RADOS objects Resolved
Copied to fs - Backport #24863: mimic: ceph_volume_client: allow atomic update of RADOS objects Resolved

History

#1 Updated by Ramana Raja 7 months ago

Greg Farnum's suggestions to do atomic RADOS object updates,

"If you've already got code that does all these things and just want to make sure it's not racy, there are a few options. Basically, you need a version, and on every write you make sure the version is what you expect. The basic outline is
1) read the object (as I assume you already do), along with its version
2) update the object in-memory, and then write it out using our compound operations. (In the C++ code these are ObjectOperations, or for updates an ObjectWriteOperation. I saw a rados_write_op_t in the python bindings which I suspect is the same) The operation will be two pieces, 1) assert that the version is as expected (and fail if not), then 2) do the write.
3) If the operation failed because the version is wrong, loop back to 1 and try again.

There are two options for getting a version ID. The first is that every object in RADOS actually has a version which is updated on every op, and you can just use that. In this case, on the read reply you'd invoke get_version() on the read completion, and then invoke assert_version() on the write op.
These versions are nice because you don't have to maintain them, but while they're unique and monotonic they will definitely not be sequential, and you're stuck with just the integer value.

The other option is to maintain a version yourself in an object xattr. In that case, you'd make sure on reads to get the xattr (not just the object data), and then on the write operation you'd cmp_xattr("my_xattr", CEPH_OSD_CMPXATTR_OP_EQ, bufferlist_of_previous_value) as the first operation, and then as your last operation you'd do setxattr("my_xattr", bufferlist_of_new_value). If you want to maintain the Ganesha export IDs in a specific format or sequence, this would be the way to go. (And indeed it's just those two operations.)
(Those are all the C++ function names, btw — I imagine you can work out the python interfaces with those keywords but I haven't used it at all.) "

I think the first option is easier to implement.

#2 Updated by Patrick Donnelly 7 months ago

  • Target version set to v14.0.0
  • Backport set to mimic,luminous
  • Severity changed from 3 - minor to 5 - suggestion
  • Labels (FS) task(intern) added

#3 Updated by Rishabh Dave 6 months ago

  • Status changed from New to In Progress

#4 Updated by Patrick Donnelly 5 months ago

  • Status changed from In Progress to Pending Backport

#5 Updated by Patrick Donnelly 5 months ago

  • Copied to Backport #24862: luminous: ceph_volume_client: allow atomic update of RADOS objects added

#6 Updated by Patrick Donnelly 5 months ago

  • Copied to Backport #24863: mimic: ceph_volume_client: allow atomic update of RADOS objects added

#7 Updated by Nathan Cutler 3 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF