Bug #20463
closedrgw: AWSv4 encoding/signature problem, can happen with listobjects marker.
0%
Description
For buckets containing object names that contain / and %, it is
possible in some circumstances (at least with boto) for "listobjects"
operations to attempt to fetch additional objects using 'marker=' and
a value containing both / and %. When this happens, when using AWSv4,
radosgw then returns a signature validation error.
It is not actually necessary have a bucket with lots of strange objects
to order to reproduce this. It can be more simply done on any bucket
by constructing a marker value.
Following is a way to do this with boto.
export S3_USE_SIGV4=1
python
import boto
import boto.s3.connection
access_key = '40JV0D93WB9U33YNQ0L4'
secret_key = '8GjwnD4iocK3gUuOAbIj9G/9QFeXgIt+n9rJk06O'
conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = 'hybodus', port = 80,
is_secure=False, calling_format = boto.s3.connection.OrdinaryCallingFormat(), )
conn._auth_handler.region_name = "default"
bucket = conn.get_bucket('my-new-bucket')
#these fail:
s=bucket.get_all_keys(marker='level1/8e%25%25FAH3')
s=bucket.get_all_keys(marker='level18e%25/FAH3')
#these work:
s=bucket.get_all_keys(marker='level1/3')
s=bucket.get_all_keys(marker='level18e%25FAH3')
I'm including a patch to ceph here that makes the above work.
Files
Updated by Marcus Watts almost 7 years ago
I've made this PR for ceph: https://github.com/ceph/ceph/pull/15998 - contains just the patch I posted above.
Updated by Marcus Watts almost 7 years ago
Jewel has the same behavior - so yes it probably needs the same fix.
Updated by Matt Benjamin over 6 years ago
jewel backport at: https://github.com/ceph/ceph/pull/16456
Updated by Matt Benjamin over 6 years ago
- Status changed from 7 to Pending Backport
- Backport set to jewel kraken
Updated by Nathan Cutler over 6 years ago
- Copied to Backport #20824: jewel: rgw: AWSv4 encoding/signature problem, can happen with listobjects marker. added
Updated by Nathan Cutler over 6 years ago
- Copied to Backport #20825: kraken: rgw: AWSv4 encoding/signature problem, can happen with listobjects marker. added
Updated by Nathan Cutler about 6 years ago
- Status changed from Pending Backport to Resolved