Actions
Bug #50415
closeddirectories with names starting with a non-ascii character disappear after reshard
% Done:
0%
Source:
Tags:
Backport:
pacific,octopus,nautilus
Regression:
No
Severity:
2 - major
Reviewed:
Description
If a directory name contains only traditional Chinese characters, after reshard the directory and its content is not seen in listing. The objects are still accessible with "get". If the directory contains at least one ascii character at the beginning it does not disappear after reshard.
Steps to reporoduce.
Deploy a vstart.sh cluster with rgw, create s3cmd config and make a test bucket:
adonis:~/ceph/ceph.ci/build% RGW=1 ../src/vstart.sh -n ... adonis:~/ceph/ceph.ci/build% cat >s3config <<EOF [default] host_base = localhost:8000 access_key = 0555b35654ad1656d804 secret_key = h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q== bucket_location = us-east-1 check_ssl_certificate = True check_ssl_hostname = True default_mime_type = binary/octet-stream delete_removed = False dry_run = False enable_multipart = True encoding = UTF-8 encrypt = False follow_symlinks = False force = False guess_mime_type = True host_bucket = anything.with.three.dots multipart_chunk_size_mb = 15 multipart_max_chunks = 10000 recursive = False recv_chunk = 65536 send_chunk = 65536 signature_v2 = False socket_timeout = 300 use_https = False use_mime_magic = True verbosity = WARNING EOF adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config mb s3://s3cmd-demo
Put files in a directory with ascii chars:
adonis:~/ceph/ceph.ci/build% mkdir /tmp/test adonis:~/ceph/ceph.ci/build% echo 1 > /tmp/test/1 adonis:~/ceph/ceph.ci/build% echo 2 > /tmp/test/2 adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config put --recursive /tmp/test/ s3://s3cmd-demo/test/ upload: '/tmp/test/1' -> 's3://s3cmd-demo/test/1' [1 of 2] 2 of 2 100% in 1s 1.03 B/s done upload: '/tmp/test/2' -> 's3://s3cmd-demo/test/2' [2 of 2] 2 of 2 100% in 0s 43.98 B/s done
Put the files in a directory with chinese chars:
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config put --recursive /tmp/test/ s3://s3cmd-demo/衛照存放區/ upload: '/tmp/test/1' -> 's3://s3cmd-demo/衛照存放區/1' [1 of 2] 2 of 2 100% in 0s 38.68 B/s done upload: '/tmp/test/2' -> 's3://s3cmd-demo/衛照存放區/2' [2 of 2] 2 of 2 100% in 0s 47.38 B/s done
Check that you can see the directory and its content in the listing:
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config ls s3://s3cmd-demo/
DIR s3://s3cmd-demo/test/
DIR s3://s3cmd-demo/衛照存放區/
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config ls s3://s3cmd-demo/test/
2021-04-19 03:21 2 s3://s3cmd-demo/test/1
2021-04-19 03:21 2 s3://s3cmd-demo/test/2
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config ls s3://s3cmd-demo/衛照存放區/
2021-04-19 03:21 2 s3://s3cmd-demo/衛照存放區/1
2021-04-19 03:21 2 s3://s3cmd-demo/衛照存放區/2
Reshard the bucket:
adonis:~/ceph/ceph.ci/build% radosgw-admin bucket reshard --bucket s3cmd-demo --num-shards 100 tenant: bucket name: s3cmd-demo old bucket instance id: 2d508e23-f38c-47ed-bd68-7ebb21d7d7c5.4411.1 new bucket instance id: 2d508e23-f38c-47ed-bd68-7ebb21d7d7c5.4441.1 total entries: 2 2021-04-19T04:23:58.428+0100 7fcacf75fb80 1 execute INFO: reshard of bucket "s3cmd-demo" from "s3cmd-demo:2d508e23-f38c-47ed-bd68-7ebb21d7d7c5.4411.1" to "s3cmd-demo:2d508e23-f38c-47ed-bd68-7ebb21d7d7c5.4441.1" completed successfully
Observe that the directory with Chinese characters only is not seen in the listing any more:
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config ls s3://s3cmd-demo/
DIR s3://s3cmd-demo/test/
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config ls s3://s3cmd-demo/test/
2021-04-19 03:21 2 s3://s3cmd-demo/test/1
2021-04-19 03:21 2 s3://s3cmd-demo/test/2
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config ls s3://s3cmd-demo/衛照存放區/
adonis:~/ceph/ceph.ci/build%
You still can access the objects with "get":
adonis:~/ceph/ceph.ci/build% s3cmd --config=s3config get s3://s3cmd-demo/衛照存放區/1 download: 's3://s3cmd-demo/衛照存放區/1' -> './1' [1 of 1] 2 of 2 100% in 0s 60.56 B/s done
Actions