Actions
Bug #985
closedradosgw listing keys of a bucket with name >=251 characters fails with NoSuchKey
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
from s3-tests:
def test_bucket_list_long_name(): length = 251 num = length - len(prefix) bucket = s3.main.create_bucket('{prefix}{name}'.format( prefix=prefix, name=num*'a', )) got = bucket.list() # rgw bug makes this fail with NoSuchKey when length >=251 got = list(got) eq(got, [])
$ S3TEST_CONF=tv.conf ./virtualenv/bin/nosetests test_s3:test_bucket_list_long_name -s Cleaning buckets from connection alt Cleaning buckets from connection main Done with cleanup of test buckets. ECleaning buckets from connection alt Cleaning buckets from connection main Cleaning bucket <Bucket: test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa> GOT UNWANTED ERROR NoSuchKey E ====================================================================== ERROR: test_s3.test_bucket_list_long_name ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/home/tv/src/s3-tests.git/test_s3.py", line 352, in test_bucket_list_long_name got = list(got) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucketlistresultset.py", line 30, in bucket_lister delimiter=delimiter, headers=headers) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 318, in get_all_keys '', headers, **params) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 285, in _get_all response.status, response.reason, body) S3ResponseError: S3ResponseError: 404 Not Found <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error> -------------------- >> begin captured logging << -------------------- boto: DEBUG: path=/ boto: DEBUG: auth_path=/ boto: DEBUG: Canonical: GET Tue, 05 Apr 2011 18:01:23 GMT / boto: DEBUG: Method: GET boto: DEBUG: Path: / boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:23 GMT', 'Content-Length': '0', 'Authorization': 'AWS TKKZ1DX83O7ZCTWHE0YD:OmG3IOkioSt3IS0TTjQd3tmyC5U=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: establishing HTTP connection boto: DEBUG: path=/ boto: DEBUG: auth_path=/ boto: DEBUG: Canonical: GET Tue, 05 Apr 2011 18:01:23 GMT / boto: DEBUG: Method: GET boto: DEBUG: Path: / boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:23 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:maX76HOJIuMLtYdFYDCy2Btynvg=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: establishing HTTP connection boto: DEBUG: path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: auth_path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Canonical: PUT Tue, 05 Apr 2011 18:01:23 GMT /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Method: PUT boto: DEBUG: Path: /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:23 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:Ju3e9a8JHb5y38yPO8rhx/3o4Xs=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: auth_path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Canonical: GET Tue, 05 Apr 2011 18:01:24 GMT /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Method: GET boto: DEBUG: Path: /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:Feuq3/DY5ODJqwyCPYT6z91Ujkk=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error> --------------------- >> end captured logging << --------------------- ====================================================================== ERROR: test suite for <module 'test_s3' from '/home/tv/src/s3-tests.git/test_s3.py'> ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/suite.py", line 227, in run self.tearDown() File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/suite.py", line 350, in tearDown self.teardownContext(ancestor) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/suite.py", line 366, in teardownContext try_run(context, names) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/util.py", line 478, in try_run return func() File "/home/tv/src/s3-tests.git/test_s3.py", line 147, in teardown nuke_prefixed_buckets() File "/home/tv/src/s3-tests.git/test_s3.py", line 57, in nuke_prefixed_buckets for key in bucket.list(): File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucketlistresultset.py", line 30, in bucket_lister delimiter=delimiter, headers=headers) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 318, in get_all_keys '', headers, **params) File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 285, in _get_all response.status, response.reason, body) S3ResponseError: S3ResponseError: 404 Not Found <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error> -------------------- >> begin captured logging << -------------------- boto: DEBUG: path=/ boto: DEBUG: auth_path=/ boto: DEBUG: Canonical: GET Tue, 05 Apr 2011 18:01:24 GMT / boto: DEBUG: Method: GET boto: DEBUG: Path: / boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS TKKZ1DX83O7ZCTWHE0YD:YRzXSPawwRU0/yrZQH1d7NxrWmg=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: path=/ boto: DEBUG: auth_path=/ boto: DEBUG: Canonical: GET Tue, 05 Apr 2011 18:01:24 GMT / boto: DEBUG: Method: GET boto: DEBUG: Path: / boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:PYM5X/79nR16aHa2Dieu5XP7TzM=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: auth_path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Canonical: GET Tue, 05 Apr 2011 18:01:24 GMT /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Method: GET boto: DEBUG: Path: /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ boto: DEBUG: Data: boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:Feuq3/DY5ODJqwyCPYT6z91Ujkk=', 'User-Agent': 'Boto/2.0b4 (linux2)'} boto: DEBUG: Host: localhost:7280 boto: DEBUG: <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error> --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 0.989s FAILED (errors=2)
Updated by Sage Weil about 13 years ago
- Translation missing: en.field_position set to 1
Updated by Colin McCabe about 13 years ago
This is a problem with long pool names in the Object Storage layer.
2011-04-12 17:39:57.797442 7f137f892700 filestore(/data/osd0) touch /data/osd0/current/4.2_head/ jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj_head = -36
This also affects non-RGW code, of course.
I think we're going to solve it at the OSD layer, by moving to creating pool directories by pool id rather than pool name.
Updated by Colin McCabe about 13 years ago
- Assignee set to Yehuda Sadeh
I added too-long-object-name error checking to rados with commit:694c4379ceb892a78a9e0f63bfe11070eff53a84 . Yehuda is going to extend the rados object name limit for the full solution.
Updated by Sage Weil about 13 years ago
- Translation missing: en.field_story_points set to 0
- Translation missing: en.field_position deleted (
7) - Translation missing: en.field_position set to 7
Updated by Sage Weil about 13 years ago
- Translation missing: en.field_story_points changed from 0 to 1
- Translation missing: en.field_position deleted (
7) - Translation missing: en.field_position set to 7
Updated by Yehuda Sadeh about 13 years ago
- Status changed from New to Resolved
Fixed, commit:95cea34e8d813236064c2d975e870c83e9224e00.
Updated by John Spray over 6 years ago
- Project changed from Ceph to rgw
- Category deleted (
22) - Target version deleted (
v0.28)
Bulk reassign of radosgw category to RGW project.
Actions