Bug #50967
closed
In master-branch rgw enviroment,The bucket list operation displays the wrong information when the upload of the fragment is over.
Added by ren weiguo almost 3 years ago.
Updated over 2 years ago.
Description
The reslut:
s3cmd mb s3://testbucket --host 0.0.0.0:8001 --access_key user1 --secret_key user1
dd if=/dev/zero of=30m bs=3M count=10
s3cmd put 30m s3://testbucket --host 0.0.0.0:8001 --access_key user1 --secret_key user1
upload: '30m' -> 's3://testbucket/30m' [part 1 of 2, 15MB] [1 of 1]
15728640 of 15728640 100% in 5s 2.61 MB/s done
upload: '30m' -> 's3://testbucket/30m' [part 2 of 2, 15MB] [1 of 1]
15728640 of 15728640 100% in 2s 6.15 MB/s done
s3cmd ls s3://testbucket --host 0.0.0.0:8001 --access_key user1 --secret_key user1
2021-05-25 06:48 31457280 s3://testbucket/30m
2021-05-25 06:48 15728640 s3://testbucket/zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1
2021-05-25 06:48 15728640 s3://testbucket/zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.2
Reproduce:
1 ./bin/radosgw-admin zone set < zone
2 ./bin/radosgw-admin zonegroup set < zonegroup
3 start rgw instance use this zone.
4 ./bin/radosgw-admin user create --uid user1 --access-key user1 --secret-key user1 --rgw-zone default_zone --rgw-zonegroup default_zonegroup --display-name "user1"
5. multipart upload object
Files
- Project changed from Ceph to rgw
- Status changed from New to Fix Under Review
- Pull request ID set to 41522
Some of us had a discussion of this topic more generally.
It's not clear that changing the name of a zone or zone group would work since their ids are stored elsewhere.
Additionally, on creation, perhaps names that contain an underscore should be rejected.
I think this will be the topic of further discussion to see what the safest path forward is.
Would you mind adding a comment that shows the output that is produced and then the output that should be produced instead?
Sorry for the late reply.From the code point of view, the problem occurs like this。
The object uploaded as above is '30m'.The name stored in the datapool is 'default_zone.4253.1_30m'.One of the shard object names is 'default_zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1'. When uploading fragments, 'MultipartObjectProcessor::prepare_head' will call the function 'raw_obj_to_obj' when processing 'stripe_obj'. This function will take apart 'default_zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1', the result we expect is
head_obj.key->name is '2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1'
head_obj.key->ns is 'multipart', But the result is head_obj.key->name = 'zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1',ns='',After In the 'RGWRados::cls_obj_prepare_op function', 'cls_rgw_obj_key key(obj.key.get_index_key_name(), obj.key.instance)' This key is translated into key.name='zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1',We expects the result should actually be '_multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1' 'zone.4253.1__multipart_30m.2~8WCdQ9lFHRsG1iZkEdPV5P0BM2jZasa.1' is then written into the index pool.
So the solution I think is raw_obj_to_obj correctly splits name, ns.
If you use the zone create command, the zone id with underscore will not be created , so there will be no such problems. You can see.'RGWSystemMetaObj::create',zone_id is boost::uuids.
- Status changed from Fix Under Review to Resolved
Also available in: Atom
PDF