Project

General

Profile

Bug #23100

admin/metadata/bucket: weird bucket name in listing, cannot fetch

Added by Robin Johnson over 2 years ago. Updated 7 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

listing of bucket names via the admin/metadata/bucket API shows this name for a bucket 2016-02-26 03:08:16.935351. Trying to fetch it shows NoSuchKey.

It's possible it was created via the Swift API which has had much looser naming rules than the S3 endpoints, but it should still not have failed to display metadata via API or command.

rjohnson@peon5752:~$ sudo radosgw-admin metadata get bucket:'2016-02-26 03:08:16.935351'
ERROR: can't get key: (2) No such file or directory
rjohnson@peon5752:~$ sudo radosgw-admin --bucket '2016-02-26 03:08:16.935351' bucket stats
{
    "bucket": "2016-02-26 03:08:16.935351",
    "zonegroup": "default",
    "placement_rule": "",
    "explicit_placement": {
        "data_pool": ".rgw.data.1",
        "data_extra_pool": "",
        "index_pool": ".rgw.data.1" 
    },
    "id": "default.197251264.11511",
    "marker": "default.197251264.11511",
    "index_type": "Normal",
    "owner": "CENSORED",
    "ver": "0#1",
    "master_ver": "0#0",
    "mtime": "2016-02-26 03:08:16.000000",
    "max_marker": "0#",
    "usage": {},
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1024,
        "max_size_kb": 0,
        "max_objects": -1
    }
}

History

#1 Updated by Robin Johnson over 2 years ago

I have also reproduced this on a new Jewel-era cluster (since Congress is pre-Argonaut and sometimes causes confusion).

#2 Updated by Orit Wasserman about 2 years ago

  • Assignee set to Yehuda Sadeh

#3 Updated by Kellen Renshaw 7 months ago

Encountered this issue in Luminous, traced it to the colon in the bucket name causing parsing issues in the metadata code, specifically in the parse_bucket() function:
https://github.com/ceph/ceph/blob/47514d5154e1b29494330a1c2fc0b26fd5cb843e/src/rgw/rgw_bucket.cc#L62

These buckets are created by (and legal according to the spec of) the Swift API.

There are a number of places [0][1][2] where the code assumes that the presence of two ':'s means that the metadata string contains a shard id. This assumption is broken when the bucket name can contain a ':'.

Ideally, the fix would be to fully encode3 the metadata keys, although that would likely require a break from the current/legacy CLI tools options.

Since that looks like a major change, it might be prudent to explicitly disallow bucket names with ':' being created via the Swift API as an interim solution.

[0] https://github.com/ceph/ceph/blob/master/src/rgw/services/svc_bucket_sobj.cc#L113
[1] https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L186
[2] https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L218
[3] https://github.com/ceph/ceph/blob/dc31b030ea30f3b7bfdeb9fa92e303892d17138a/src/rgw/rgw_common.h#L1194

Also available in: Atom PDF