Bug #23100
openadmin/metadata/bucket: weird bucket name in listing, cannot fetch
0%
Description
listing of bucket names via the admin/metadata/bucket API shows this name for a bucket 2016-02-26 03:08:16.935351. Trying to fetch it shows NoSuchKey.
It's possible it was created via the Swift API which has had much looser naming rules than the S3 endpoints, but it should still not have failed to display metadata via API or command.
rjohnson@peon5752:~$ sudo radosgw-admin metadata get bucket:'2016-02-26 03:08:16.935351'
ERROR: can't get key: (2) No such file or directory
rjohnson@peon5752:~$ sudo radosgw-admin --bucket '2016-02-26 03:08:16.935351' bucket stats
{
"bucket": "2016-02-26 03:08:16.935351",
"zonegroup": "default",
"placement_rule": "",
"explicit_placement": {
"data_pool": ".rgw.data.1",
"data_extra_pool": "",
"index_pool": ".rgw.data.1"
},
"id": "default.197251264.11511",
"marker": "default.197251264.11511",
"index_type": "Normal",
"owner": "CENSORED",
"ver": "0#1",
"master_ver": "0#0",
"mtime": "2016-02-26 03:08:16.000000",
"max_marker": "0#",
"usage": {},
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1024,
"max_size_kb": 0,
"max_objects": -1
}
}
Updated by Robin Johnson about 6 years ago
I have also reproduced this on a new Jewel-era cluster (since Congress is pre-Argonaut and sometimes causes confusion).
Updated by Kellen Renshaw over 4 years ago
Encountered this issue in Luminous, traced it to the colon in the bucket name causing parsing issues in the metadata code, specifically in the parse_bucket() function:
https://github.com/ceph/ceph/blob/47514d5154e1b29494330a1c2fc0b26fd5cb843e/src/rgw/rgw_bucket.cc#L62
These buckets are created by (and legal according to the spec of) the Swift API.
There are a number of places [0]12 where the code assumes that the presence of two ':'s means that the metadata string contains a shard id. This assumption is broken when the bucket name can contain a ':'.
Ideally, the fix would be to fully encode3 the metadata keys, although that would likely require a break from the current/legacy CLI tools options.
Since that looks like a major change, it might be prudent to explicitly disallow bucket names with ':' being created via the Swift API as an interim solution.
[0] https://github.com/ceph/ceph/blob/master/src/rgw/services/svc_bucket_sobj.cc#L113
[1] https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L186
[2] https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L218
[3] https://github.com/ceph/ceph/blob/dc31b030ea30f3b7bfdeb9fa92e303892d17138a/src/rgw/rgw_common.h#L1194