Ceph

h1. RESTful API for DR  Geo-Replication

h1. RESTful API for DR  Geo-Replication

h3. System User Special Requests

In general, a system user (a user that has the system flag set on) may operate on behalf of other users. Certain requests may either have special request params (always prefixed by 'rgwx-') or may have special header fields (prefixed by 'Rgwx-').

h4. Params:

* rgwx-uid

The effective user id for this request

h4. Create bucket for a non-master region

Tells the master region that a user tries to create a bucket. Can only be executed by a system user. The original bucket creation request is being forwarded under a different user context, with some extra params.

Syntax:

PUT /bucket

Extra Params:

1. rgwx-region: source region for request

2. rgwx-uid: effective user id for bucket creation

Response:

Response data for a successful bucket creation returns the JSON encoded bucket object metadata version and the bucket info.

h4. Extended object GET

Read an object, send extra metadata about object. This request extends generic object read.

Syntax

GET /bucket/object

Extra Params:

1. rgwx-prepend-metadata: whether object metadata will be prepended to object data as part of the response.

Extra Response Headers:

1.  Rgwx-Embedded-Metadata-Len: Length of prepended metadata (if requested)

h4. Extended object PUT

Writes an object, return extra object metadata. Extends regular object write.

Syntax

PUT /bucket/object

Extra Response Headers:

1. Rgwx-Mtime: Object modification time.

*Sync object from zone within region (intra-region copy)*

Syntax:

PUT /bucket/objcect

Extra Params:

1. rgwx-source-zone: the source zone for where 

2. rgwx-client-id: the id of the agent that initiated the request

3. rgwx-op-od: a unique op id, created by the agent

It is up for the agent to provide a meaningful client id and a unique (for that agent) op id. These will be used for handling opstate information.

%{color:yellow}Metadata%

h4. List metadata

Display list of metadata sections, keys.

Syntax:

GET /admin/metadata[/section]

Response:

JSON encoded list of sections or keys.

Examples:

1. GET /admin/metadata

[

    "user"]

2. GET /admin/metadata/user

[

    "yehsad",

    "greg"]

h4. Get metadata info

Syntax:

GET /admin/metadata/<section>?key=<key>

Response:

JSON encoded metadata info

Examples:

GET /admin/metadata/user?key=yehsad

{ "key": "user:yehsad",

  "ver": { "ver": 3,

      "tag": "8L0en1nFM1Y+NaQrBAzgPliZ"},

  "data": { "user_id": "yehsad",

      "display_name": "yehuda",

      "email": "",

      "suspended": 0,

      "max_buckets": 1000,

      "auid": 0,

      "subusers": [

            { "id": "yehsad:yehsad",

              "permissions": "<none>"}],

      "keys": [

            { "user": "yehsad",

              "access_key": "foo",

              "secret_key": "bar"}],

      "swift_keys": [

            { "user": "yehsad:yehsad",

              "secret_key": "zxczxc"}],

      "caps": []}}

h4. Set metadata info

Syntax:

PUT /admin/metadata/<section>?key=<key>

<JSON encoded metadata>

Response:

Examples:

PUT /admin/metadata/user?key=yehsad

{ "key": "user:yehsad",

  "ver": { "ver": 4,

      "tag": "8L0en1nFM1Y+NaQrBAzgPliZ"},

  "data": { "user_id": "yehsad",

      "display_name": "yehuda",

      "email": "",

      "suspended": 0,

      "max_buckets": 10,

      "auid": 0,

      "subusers": [

            { "id": "yehsad:yehsad",

              "permissions": "<none>"}],

      "keys": [

            { "user": "yehsad",

              "access_key": "foo",

              "secret_key": "bar"}],

      "swift_keys": [

            { "user": "yehsad:yehsad",

              "secret_key": "zxczxc"}],

      "caps": []}}

h4. Remove metadata

Remove a metadata key

Syntax:

DELETE /admin/metadata/<section>?key=<key>

Response:

Example:

DELETE /admin/metadata/user?key=yehsad

h3. Data, Metadata Logs

get log info

Returns info about the log. Currently that only includes the number of log shards.

Syntax:

GET /admin/log?type=<type>

params:

1. type: metadata, data

result:

return a JSON representation of the data log information

{

  num_objects = <num>

}

h4. ​get shard info

Retuns information about the current log shard state. Shard id is in the range of 0 to num_objects - 1 (num_objects can be retrieved by the get log info request).

Syntax:

GET /admin/log?info&type=<type>&id=<shard-id>

params:

1. type: metadata, data

2. id: shard id

result:

return a JSON representation of the data log information

{

  num_objects = <num>

}

h4. Lock log shard

POST /admin/log?lock

params:

1. type: metadata, data

2. id: shard id (0..num_objects)

3. length: time in seconds for lock to be valid

4. zone-id: locker zone id

5. locker-id: client id (e.g., agent id).

Returns 404 if there are no entries of the specified type in the shard.

h4. Unlock log shard

POST /admin/log?unlock

params:

1. type: metadata, data

2. id: shard id (0..num_objects)

3. zone-id: locker zone id

4. locker-id: unique user identifier that corresponds to a previous lock call

h4. list log

GET /admin/log?id=<shard id>

params:

1. type: metadata, data

2. id=<shard id>

3. start-time=<start time> (optional)

4. end-time=<end time> (optional)

5. marker=<start marker> (optional)

Need to have either one pair of start-time and end-time specified, or start-marker and end-marker specified. Markers supersede timestamps, and they correspond to the 'id' field in the log entries.

h4. trim data log

DELETE /admin/log?id=<shard id>

params:

1. type: metadata, data

2. id=<shard id>

3. start-time=<start time>

4. end-time=<end time>

5. start-marker=<start marker>

6. end-marker=<end marker> 

Need to have either one pair of start-time and end-time specified, or start-marker and end-marker specified. Markers supersede timestamps, and they correspond to the 'id' field in the log entries.

h3. Bucket Index Log

h4. List bucket index log

GET /admin/log?type=bucket-index

params:

1. bucket=<bucket name>

2. bucket-instance=<bucket instance id>

3. start-marker=<start-marker>

4. end-marker=<end-marker>

Notes:

 - One of either bucket or bucket-instance must be specified.

 - The marker param can be used to iterate through entries. It corresponds to the "id" field in a returned entry.

h4. Get bucket index log info

Get information about bucket index log state. Can be used to retrieve current last log entry's marker position.

GET /admin/log?info&type=bucket-index

params:

1. bucket=<bucket name>

2. bucket-instance=<bucket instance id>Notes:

 - One of either bucket or bucket-instance must be specified.

h4. trim bucket index log

DELETE /admin/log?type=bucket-index

params:

1. bucket=<bucket name>

2. bucket-instance=<bucket instance>

3. marker=<marker> (optional)

4. max-entries=<max entries> (optional, default is 1000)

Notes:

 - One of either bucket or bucket-instance must be specified.

h3. Replica Log

h4. Data/Metadata Replica Log

h4. Set a Worker Bound

POST /admin/replica_log?work_bound

params:

1) type: metadata, data

2) id: shard id

3) marker: as used for log listing

4) time: the lower bound master zone time (corresponds to the marker)

5) daemon_id: a user-provided string, which can be anything.

Currenly there can only be one worker bound stored for a data or metadata shard, and there are a few constraints on setting it:

daemon_id must be the same as the existing entry, if there is one

time must not be before the existing timestamp, if there is one

marker must not be empty

body:

the body of the request includes a JSON-encoded list of buckets which are "in progress" from before the lower bound, plus a lower bound on the time associated with each of them:

[ { "bucket": <name>, "time": <time> }, { "bucket": <name2>, "time": <time2> } ]

h4. Get the Worker bounds

GET /admin/replica_log?bounds

params:

1) type: metadata, data, bucket-index (see below)

2) id: shard id

This returns a JSON object containing a list of the lower bounds and bucket collections for each daemon_id, plus the aggregate's lowest marker and oldest time bound:

{

  "markers":

  [ { "entity": <id1>, "position_marker": <marker1>, "postion_time": <time1>, "items_in_progress": [ { "name": <name>, "timestamp":2 <time2> }, { "name": <name2>, "timestamp": <time3> } ] },

    { "entity": <id2>, "positon_marker": <marker2>, "position_time": <time4>, "items_in_progress": [ ] }

  ],

  "marker": <marker1>,

  "oldest_time": <time3>

}

h4. Remove a daemon's bound

DELETE /admin/replica_log?work_bound

params:

1) type: metadata, data, bucket-index (see below)

2) id: shard id

3) daemon_id: a user-provided string, which can be anything

h3. Bucket Index Replica Log

h4. Set a Worker Bound

POST /admin/replica_log?work_bound

params:

1) type=bucket-index

2) bucket-instance: bucket instance id

3) marker: as used for bucket log listing

4) time: the lower bound master zone time for this bucket (corresponds to the marker)

5) daemon_id: a user-provided string, which can be anything

Currenly there can only be one worker bound stored for a bucket-instance, and there are a few constraints on setting it:

daemon_id must be the same as the existing entry, if there is one

time must not be before the existing timestamp, if there is one

marker must not be empty

The request body includes a JSON list of objects in progress, which includes the time each of them became out-of-date:

[ { name:"objfoo", time:<time1>}, { name:"objbar", time:<time2>}, { name:"objbaz", time:<time3>} ]

h4. Get a Worker Bound

GET /admin/replica_log?bounds

params:

1) type=bucket-index

2) bucket-instance: bucket instance id

It returns an object containing the post data and the params from the last call for each daemon, plus an aggregate lowest bound marker and oldest_time

{

  "markers": [ { "entity": <id1>, "position_marker": <marker1>, "position_time": <time1>, "items_in_progress": [ { name:"objfoo", time:<time1>}, { name:"objbar", time:<time2>}, { name:"objbaz", time:<time3>} ],

    { "entity": <id2>, "position_marker": <marker2>, "position_time": <time2>, "items_in_progress": [] } ],

  "marker": <marker1>,

  "oldest_time": <time3>

}

h4. Delete a Worker's Bound

DELETE /admin/replica_log?work_bound

params:

1) type: bucket-index

2) bucket-instance: bucket instance id

3) daemon_id: a user-provided string, which can be anything

h3. OpState

h4. List entries

GET /admin/opstate

params:

1. client-id=<client-id> (optional)

2. op-id=<op-id> (optional)

3. object=<object> (optional)

4. max-entries=<max entries> (optional, default is 1000)

h4. Set entry

POST /admin/opstate

params:

1. client-id=<client-id> (required)

2. op-id=<op-id> (required)

3. object=<object> (requiredl)

4. state=<state> (required)

5. renew (optional)

if renew is specified then operation can only succeed if the state specified matches the existing state.

h4. Remove entry

DELETE /admin/opstate

params:

1. client-id=<client-id> (required)

2. op-id=<op-id> (required)

3. object=<object> (requiredl)

h3. Reading Configuration

At the moment, region configuration information is read-only.

h4. Get region map

Syntax:

GET /admin/config