Project

General

Profile

Actions
Copy link

Bug #985

closed

radosgw listing keys of a bucket with name >=251 characters fails with NoSuchKey

Added by Anonymous about 13 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Yehuda Sadeh
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

from s3-tests:

def test_bucket_list_long_name():
    length = 251
    num = length - len(prefix)
    bucket = s3.main.create_bucket('{prefix}{name}'.format(
            prefix=prefix,
            name=num*'a',
            ))
    got = bucket.list()
    # rgw bug makes this fail with NoSuchKey when length >=251
    got = list(got)
    eq(got, [])


$ S3TEST_CONF=tv.conf ./virtualenv/bin/nosetests test_s3:test_bucket_list_long_name -s
Cleaning buckets from connection alt
Cleaning buckets from connection main
Done with cleanup of test buckets.
ECleaning buckets from connection alt
Cleaning buckets from connection main
Cleaning bucket <Bucket: test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa>
GOT UNWANTED ERROR NoSuchKey
E
======================================================================
ERROR: test_s3.test_bucket_list_long_name
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/case.py", line 187, in runTest
    self.test(*self.arg)
  File "/home/tv/src/s3-tests.git/test_s3.py", line 352, in test_bucket_list_long_name
    got = list(got)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucketlistresultset.py", line 30, in bucket_lister
    delimiter=delimiter, headers=headers)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 318, in get_all_keys
    '', headers, **params)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 285, in _get_all
    response.status, response.reason, body)
S3ResponseError: S3ResponseError: 404 Not Found
<?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error>
-------------------- >> begin captured logging << --------------------
boto: DEBUG: path=/
boto: DEBUG: auth_path=/
boto: DEBUG: Canonical: GET

Tue, 05 Apr 2011 18:01:23 GMT
/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:23 GMT', 'Content-Length': '0', 'Authorization': 'AWS TKKZ1DX83O7ZCTWHE0YD:OmG3IOkioSt3IS0TTjQd3tmyC5U=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: establishing HTTP connection
boto: DEBUG: path=/
boto: DEBUG: auth_path=/
boto: DEBUG: Canonical: GET

Tue, 05 Apr 2011 18:01:23 GMT
/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:23 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:maX76HOJIuMLtYdFYDCy2Btynvg=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: establishing HTTP connection
boto: DEBUG: path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: auth_path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Canonical: PUT

Tue, 05 Apr 2011 18:01:23 GMT
/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:23 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:Ju3e9a8JHb5y38yPO8rhx/3o4Xs=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: auth_path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Canonical: GET

Tue, 05 Apr 2011 18:01:24 GMT
/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:Feuq3/DY5ODJqwyCPYT6z91Ujkk=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error>
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test suite for <module 'test_s3' from '/home/tv/src/s3-tests.git/test_s3.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/suite.py", line 227, in run
    self.tearDown()
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/suite.py", line 350, in tearDown
    self.teardownContext(ancestor)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/suite.py", line 366, in teardownContext
    try_run(context, names)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/nose/util.py", line 478, in try_run
    return func()
  File "/home/tv/src/s3-tests.git/test_s3.py", line 147, in teardown
    nuke_prefixed_buckets()
  File "/home/tv/src/s3-tests.git/test_s3.py", line 57, in nuke_prefixed_buckets
    for key in bucket.list():
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucketlistresultset.py", line 30, in bucket_lister
    delimiter=delimiter, headers=headers)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 318, in get_all_keys
    '', headers, **params)
  File "/home/tv/src/s3-tests.git/virtualenv/lib/python2.6/site-packages/boto/s3/bucket.py", line 285, in _get_all
    response.status, response.reason, body)
S3ResponseError: S3ResponseError: 404 Not Found
<?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error>
-------------------- >> begin captured logging << --------------------
boto: DEBUG: path=/
boto: DEBUG: auth_path=/
boto: DEBUG: Canonical: GET

Tue, 05 Apr 2011 18:01:24 GMT
/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS TKKZ1DX83O7ZCTWHE0YD:YRzXSPawwRU0/yrZQH1d7NxrWmg=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: path=/
boto: DEBUG: auth_path=/
boto: DEBUG: Canonical: GET

Tue, 05 Apr 2011 18:01:24 GMT
/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:PYM5X/79nR16aHa2Dieu5XP7TzM=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: auth_path=/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Canonical: GET

Tue, 05 Apr 2011 18:01:24 GMT
/test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /test-tv-xj1y6cmps3eylij3mqqqr-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
boto: DEBUG: Data: 
boto: DEBUG: Headers: {'Date': 'Tue, 05 Apr 2011 18:01:24 GMT', 'Content-Length': '0', 'Authorization': 'AWS O54XVCC9MQ9Q72TWP5Y1:Feuq3/DY5ODJqwyCPYT6z91Ujkk=', 'User-Agent': 'Boto/2.0b4 (linux2)'}
boto: DEBUG: Host: localhost:7280
boto: DEBUG: <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error>
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 0.989s

FAILED (errors=2)

Related issues 1 (0 open — 1 closed)

Blocked by Ceph - Feature #1008: filestore: handle long object namesResolvedYehuda Sadeh04/16/2011

Actions
Actions
Copy link
 #1

Updated by Sage Weil about 13 years ago

  • Target version set to v0.28
Actions
Copy link
 #2

Updated by Sage Weil about 13 years ago

  • Translation missing: en.field_position set to 1
Actions
Copy link
 #3

Updated by Colin McCabe about 13 years ago

This is a problem with long pool names in the Object Storage layer.

2011-04-12 17:39:57.797442 7f137f892700 filestore(/data/osd0) touch /data/osd0/current/4.2_head/
jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj_head = -36

This also affects non-RGW code, of course.

I think we're going to solve it at the OSD layer, by moving to creating pool directories by pool id rather than pool name.

Actions
Copy link
 #4

Updated by Colin McCabe about 13 years ago

this is related to #963

Actions
Copy link
 #5

Updated by Colin McCabe about 13 years ago

  • Assignee set to Yehuda Sadeh

I added too-long-object-name error checking to rados with commit:694c4379ceb892a78a9e0f63bfe11070eff53a84 . Yehuda is going to extend the rados object name limit for the full solution.

Actions
Copy link
 #6

Updated by Sage Weil about 13 years ago

  • Translation missing: en.field_story_points set to 0
  • Translation missing: en.field_position deleted (7)
  • Translation missing: en.field_position set to 7
Actions
Copy link
 #7

Updated by Sage Weil about 13 years ago

  • Translation missing: en.field_story_points changed from 0 to 1
  • Translation missing: en.field_position deleted (7)
  • Translation missing: en.field_position set to 7
Actions
Copy link
 #8

Updated by Yehuda Sadeh almost 13 years ago

  • Status changed from New to Resolved

Fixed, commit:95cea34e8d813236064c2d975e870c83e9224e00.

Actions
Copy link
 #9

Updated by John Spray over 6 years ago

  • Project changed from Ceph to rgw
  • Category deleted (22)
  • Target version deleted (v0.28)

Bulk reassign of radosgw category to RGW project.

Actions
Copy link

Also available in: Atom PDF