RESTful API for DR Geo-Replication¶

RESTful API for DR Geo-Replication¶

System User Special Requests¶

In general, a system user (a user that has the system flag set on) may operate on behalf of other users. Certain requests may either have special request params (always prefixed by 'rgwx-') or may have special header fields (prefixed by 'Rgwx-').

Params:¶

• rgwx-uid

The effective user id for this request

Create bucket for a non-master region¶

Tells the master region that a user tries to create a bucket. Can only be executed by a system user. The original bucket creation request is being forwarded under a different user context, with some extra params.

Syntax:

PUT /bucket

Extra Params:

1. rgwx-region: source region for request
2. rgwx-uid: effective user id for bucket creation

Response:
Response data for a successful bucket creation returns the JSON encoded bucket object metadata version and the bucket info.

Extended object GET¶

Read an object, send extra metadata about object. This request extends generic object read.

Syntax
GET /bucket/object

Extra Params:
1. rgwx-prepend-metadata: whether object metadata will be prepended to object data as part of the response.

Extra Response Headers:

1. Rgwx-Embedded-Metadata-Len: Length of prepended metadata (if requested)

Extended object PUT¶

Writes an object, return extra object metadata. Extends regular object write.

Syntax
PUT /bucket/object

Extra Response Headers:
1. Rgwx-Mtime: Object modification time.
Sync object from zone within region (intra-region copy)

Syntax:
PUT /bucket/objcect

Extra Params:
1. rgwx-source-zone: the source zone for where
2. rgwx-client-id: the id of the agent that initiated the request
3. rgwx-op-od: a unique op id, created by the agent

It is up for the agent to provide a meaningful client id and a unique (for that agent) op id. These will be used for handling opstate information.

Metadata

List metadata¶

Display list of metadata sections, keys.

Syntax:

GET /admin/metadata[/section]

Response:

JSON encoded list of sections or keys.

Examples:

1. GET /admin/metadata

[
"user"]

2. GET /admin/metadata/user

[
"yehsad",
"greg"]

Get metadata info¶

Syntax:

GET /admin/metadata/<section>?key=<key>

Response:

JSON encoded metadata info

Examples:

GET /admin/metadata/user?key=yehsad

{ "key": "user:yehsad",
"ver": { "ver": 3,
"tag": "8L0en1nFM1Y+NaQrBAzgPliZ"},
"data": { "user_id": "yehsad",
"display_name": "yehuda",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [ { "id": "yehsad:yehsad",
"permissions": "<none>"}],
"keys": [ { "user": "yehsad",
"access_key": "foo",
"secret_key": "bar"}],
"swift_keys": [ { "user": "yehsad:yehsad",
"secret_key": "zxczxc"}],
"caps": []}}

Set metadata info¶

Syntax:

PUT /admin/metadata/<section>?key=<key>

<JSON encoded metadata>

Response:

Examples:

PUT /admin/metadata/user?key=yehsad

{ "key": "user:yehsad",
"ver": { "ver": 4,
"tag": "8L0en1nFM1Y+NaQrBAzgPliZ"},
"data": { "user_id": "yehsad",
"display_name": "yehuda",
"email": "",
"suspended": 0,
"max_buckets": 10,
"auid": 0,
"subusers": [ { "id": "yehsad:yehsad",
"permissions": "<none>"}],
"keys": [ { "user": "yehsad",
"access_key": "foo",
"secret_key": "bar"}],
"swift_keys": [ { "user": "yehsad:yehsad",
"secret_key": "zxczxc"}],
"caps": []}}

Remove metadata¶

Remove a metadata key

Syntax:

DELETE /admin/metadata/<section>?key=<key>

Response:

Example:

DELETE /admin/metadata/user?key=yehsad

Data, Metadata Logs¶

get log info

Returns info about the log. Currently that only includes the number of log shards.

Syntax:

GET /admin/log?type=<type>

params:
1. type: metadata, data

result:

return a JSON representation of the data log information

{
num_objects = <num>
}

​get shard info¶

Retuns information about the current log shard state. Shard id is in the range of 0 to num_objects - 1 (num_objects can be retrieved by the get log info request).

Syntax:

GET /admin/log?info&type=<type>&id=<shard-id>

params:
1. type: metadata, data
2. id: shard id

result:

return a JSON representation of the data log information

{
num_objects = <num>
}

Lock log shard¶

POST /admin/log?lock

params:

1. type: metadata, data
2. id: shard id (0..num_objects)
3. length: time in seconds for lock to be valid
4. zone-id: locker zone id
5. locker-id: client id (e.g., agent id).

Returns 404 if there are no entries of the specified type in the shard.

Unlock log shard¶

POST /admin/log?unlock

params:

1. type: metadata, data
2. id: shard id (0..num_objects)
3. zone-id: locker zone id
4. locker-id: unique user identifier that corresponds to a previous lock call

list log¶

GET /admin/log?id=<shard id>

params:

1. type: metadata, data
2. id=<shard id>
3. start-time=<start time> (optional)
4. end-time=<end time> (optional)
5. marker=<start marker> (optional)

Need to have either one pair of start-time and end-time specified, or start-marker and end-marker specified. Markers supersede timestamps, and they correspond to the 'id' field in the log entries.

trim data log¶

DELETE /admin/log?id=<shard id>

params:

1. type: metadata, data
2. id=<shard id>
3. start-time=<start time>
4. end-time=<end time>
5. start-marker=<start marker>
6. end-marker=<end marker>

Need to have either one pair of start-time and end-time specified, or start-marker and end-marker specified. Markers supersede timestamps, and they correspond to the 'id' field in the log entries.

Bucket Index Log¶

List bucket index log¶

GET /admin/log?type=bucket-index

params:

1. bucket=<bucket name>
2. bucket-instance=<bucket instance id>
3. start-marker=<start-marker>
4. end-marker=<end-marker>
Notes:
- One of either bucket or bucket-instance must be specified.
- The marker param can be used to iterate through entries. It corresponds to the "id" field in a returned entry.

Get bucket index log info¶

Get information about bucket index log state. Can be used to retrieve current last log entry's marker position.

GET /admin/log?info&type=bucket-index

params:

1. bucket=<bucket name>
2. bucket-instance=<bucket instance id>Notes:
- One of either bucket or bucket-instance must be specified.

trim bucket index log¶

DELETE /admin/log?type=bucket-index

params:

1. bucket=<bucket name>
2. bucket-instance=<bucket instance>
3. marker=<marker> (optional)
4. max-entries=<max entries> (optional, default is 1000)

Notes:
- One of either bucket or bucket-instance must be specified.

Replica Log¶

Data/Metadata Replica Log¶

Set a Worker Bound¶

POST /admin/replica_log?work_bound

params:
1) type: metadata, data
2) id: shard id
3) marker: as used for log listing
4) time: the lower bound master zone time (corresponds to the marker)
5) daemon_id: a user-provided string, which can be anything.

Currenly there can only be one worker bound stored for a data or metadata shard, and there are a few constraints on setting it:
daemon_id must be the same as the existing entry, if there is one
time must not be before the existing timestamp, if there is one
marker must not be empty

body:
the body of the request includes a JSON-encoded list of buckets which are "in progress" from before the lower bound, plus a lower bound on the time associated with each of them:
[ { "bucket": <name>, "time": <time> }, { "bucket": <name2>, "time": <time2> } ]

Get the Worker bounds¶

GET /admin/replica_log?bounds
params:
1) type: metadata, data, bucket-index (see below)
2) id: shard id

This returns a JSON object containing a list of the lower bounds and bucket collections for each daemon_id, plus the aggregate's lowest marker and oldest time bound: {
"markers":
[ { "entity": <id1>, "position_marker": <marker1>, "postion_time": <time1>, "items_in_progress": [ { "name": <name>, "timestamp":2 <time2> }, { "name": <name2>, "timestamp": <time3> } ] }, { "entity": <id2>, "positon_marker": <marker2>, "position_time": <time4>, "items_in_progress": [ ] }
],
"marker": <marker1>,
"oldest_time": <time3>
}

Remove a daemon's bound¶

DELETE /admin/replica_log?work_bound
params:
1) type: metadata, data, bucket-index (see below)
2) id: shard id
3) daemon_id: a user-provided string, which can be anything

Bucket Index Replica Log¶

Set a Worker Bound¶

POST /admin/replica_log?work_bound
params:
1) type=bucket-index
2) bucket-instance: bucket instance id
3) marker: as used for bucket log listing
4) time: the lower bound master zone time for this bucket (corresponds to the marker)
5) daemon_id: a user-provided string, which can be anything

Currenly there can only be one worker bound stored for a bucket-instance, and there are a few constraints on setting it:
daemon_id must be the same as the existing entry, if there is one
time must not be before the existing timestamp, if there is one
marker must not be empty

The request body includes a JSON list of objects in progress, which includes the time each of them became out-of-date:
[ { name:"objfoo", time:<time1>}, { name:"objbar", time:<time2>}, { name:"objbaz", time:<time3>} ]

Get a Worker Bound¶

GET /admin/replica_log?bounds
params:
1) type=bucket-index
2) bucket-instance: bucket instance id

It returns an object containing the post data and the params from the last call for each daemon, plus an aggregate lowest bound marker and oldest_time {
"markers": [ { "entity": <id1>, "position_marker": <marker1>, "position_time": <time1>, "items_in_progress": [ { name:"objfoo", time:<time1>}, { name:"objbar", time:<time2>}, { name:"objbaz", time:<time3>} ], { "entity": <id2>, "position_marker": <marker2>, "position_time": <time2>, "items_in_progress": [] } ],
"marker": <marker1>,
"oldest_time": <time3>
}

Delete a Worker's Bound¶

DELETE /admin/replica_log?work_bound
params:
1) type: bucket-index
2) bucket-instance: bucket instance id
3) daemon_id: a user-provided string, which can be anything

OpState¶

List entries¶

GET /admin/opstate

params:

1. client-id=<client-id> (optional)
2. op-id=<op-id> (optional)
3. object=<object> (optional)
4. max-entries=<max entries> (optional, default is 1000)

Set entry¶

POST /admin/opstate
params:
1. client-id=<client-id> (required)
2. op-id=<op-id> (required)
3. object=<object> (requiredl)
4. state=<state> (required)
5. renew (optional)
if renew is specified then operation can only succeed if the state specified matches the existing state.

Remove entry¶

DELETE /admin/opstate
params:
1. client-id=<client-id> (required)
2. op-id=<op-id> (required)
3. object=<object> (requiredl)

Reading Configuration¶

At the moment, region configuration information is read-only.

Get region map¶

Syntax:
GET /admin/config