Ceph » rgw

It appears this is due the replica log api using the bucket name only, not the bucket-instance which is given to it by the client. Note the bucket instance is different:

2014-05-06T12:44:29.224 9331:DEBUG:requests.packages.urllib3.connectionpool:"POST /admin/replica_log?work_bound&marker=00000000004.4.3&daemon_id=radosgw-agent&type=bucket
-index&bucket-instance=myfoodata%3Ar0z0.4137.4&time=1970-01-01+00%3A00%3A00 HTTP/1.1" 200 None
...
2014-05-06T12:45:38.751 9391:DEBUG:requests.packages.urllib3.connectionpool:"GET /admin/replica_log?bounds&type=bucket-index&bucket-instance=myfoodata%3Ar0z0.4137.5 HTTP/
1.1" 200 None
2014-05-06T12:45:38.752 9391:DEBUG:boto:get_worker_bound returned: {u'marker': u'00000000004.4.3', u'oldest_time': u'0.000000', u'markers': [{u'items_in_progress': [], u'position_time': u'0.000000', u'position_marker': u'00000000004.4.3', u'entity': u'radosgw-agent'}]}
...
2014-05-06T12:45:38.757 9391:DEBUG:requests.packages.urllib3.connectionpool:"GET /admin/log?marker=00000000004.4.3&type=bucket-index&bucket-instance=myfoodata%3Ar0z0.4137.5&max-entries=10 HTTP/1.1" 200 None
2014-05-06T12:45:38.768 9391:DEBUG:radosgw_agent.worker:bucket instance "myfoodata:r0z0.4137.5" has 0 entries after "00000000004.4.3" 

radosgw-admin bilog list confirms the log entries are there:

2014-05-06 14:11:22.020572 7fa563258780 10 moving .region0.r0z0.domain_root+.bucket.meta.myfoodata:r0z0.4137.5 to cache LRU end
[
    { "op_id": "00000000001.1.2",
      "op_tag": "r0z0.4137.1074",
      "op": "write",
      "object": "tiny_file",
      "state": "pending",
      "index_ver": 1,
      "timestamp": "0.000000",
      "ver": { "pool": -1,
          "epoch": 0}},
    { "op_id": "00000000002.2.3",
      "op_tag": "r0z0.4137.1074",
      "op": "write",
      "object": "tiny_file",
      "state": "complete",
      "index_ver": 2,
      "timestamp": "2014-05-06 19:44:32.000000Z",
      "ver": { "pool": 7,
          "epoch": 1}}]

This can be reproduced by using the wip-rgw-8251 branch of teuthology with the following configuration:

interactive-on-error: true
roles:
- [mon.a, osd.0, osd.1, client.0, client.1]
tasks:
- chef:
- install:
    branch: firefly
- ceph:
    conf:
      client:
        debug ms: 1
        debug rgw: 30
        rgw gc obj min wait: 15
        rgw data log window: 30
      client.0:
        rgw region: region0
        rgw zone: r0z0
        rgw region root pool: .rgw.region.0
        rgw zone root pool: .rgw.zone.0
        rgw gc pool: .rgw.gc.0
        rgw user uid pool: .users.uid.0
        rgw user keys pool: .users.0
        rgw log data: True
        rgw log meta: True
      client.1:
        rgw region: region0
        rgw zone: r0z1
        rgw region root pool: .rgw.region.0
        rgw zone root pool: .rgw.zone.1
        rgw gc pool: .rgw.gc.1
        rgw user uid pool: .users.uid.1
        rgw user keys pool: .users.1
        rgw log data: False
        rgw log meta: False
- rgw:
    regions:
      region0:
        api name: api1
        is master: True
        master zone: r0z0
        zones: [r0z0, r0z1]
    client.0:
      system user:
        name: client0-system-user
        access key: 0te6NH5mcdcq0Tc5i8i2
        secret key: Oy4IOauQoL18Gp2zM7lC1vLmoawgqcYPbYGcWfXv
    client.1:
      system user:
        name: client1-system-user
        access key: 1te6NH5mcdcq0Tc5i8i3
        secret key: Py4IOauQoL18Gp2zM7lC1vLmoawgqcYPbYGcWfXw
- radosgw-agent:
    client.0:
      max-entries: 10
      src: client.0
      dest: client.1
- radosgw-admin:

giant is end of life