Bug #53423
openCalling list_buckets after assuming a role lists all my buckets, not their buckets
0%
Description
Hi,
I've been testing out the AssumeRole example in https://docs.ceph.com/en/latest/radosgw/STS/#examples and it seems calling the list_buckets() method on the S3 client returns a list of my buckets, rather than the buckets in the project I assumed a role in.
Using the example given on that page, after substituting in relevant credentials for two different users I see a listing of buckets for the TESTER1 user, rather than the expected list of buckets for the TESTER user. (In my case this is actually two Openstack projects, with the Principal changed to arn:aws:iam::${PROJECT_ID}:root).
Otherwise assuming a role works as expected - for example calling `list_objects_v2` on a bucket lists objects in buckets owned by TESTER, but not buckets owned by TESTER1.
I've also tried this on AWS with two accounts, and it works as you'd expect: I see a list of buckets in the resource owner's (e.g. TESTER) account, not the resource accessor (e.g. TESTER1) account.
I'm using Ceph Pacific 16.2.6, integrated with Openstack Wallaby.
Cheers
Files
Updated by Sam Mesterton-Gibbons over 2 years ago
Another, possibly related oddity: If I try and create a bucket using the assumed role I get a TooManyBuckets client error from botocore. Looking at the underlying quotas, there is plenty of quota remaining for the TESTER (resource owner) user (~1500 buckets used of 11000 quota). The TESTER1 (resource accessor) user has used 5 buckets of a quota of 1000 - is it possible the quota check is being made on the TESTER user's bucket count, with the TESTER1 user quota?
If it's unrelated I'll dig further and open another ticket.
Updated by Pritha Srivastava over 2 years ago
The AssumeRole functionality has been tested for local RGW users and not for external Openstack Keystone users. Also the format of Principal for a user : arn:aws:iam::${PROJECT_ID}:root) is not supported in RGW.
Updated by Sam Mesterton-Gibbons over 2 years ago
- File sts-docs-example-redacted.py sts-docs-example-redacted.py added
- File bucket_list_match.log bucket_list_match.log added
Pritha Srivastava wrote:
The AssumeRole functionality has been tested for local RGW users and not for external Openstack Keystone users. Also the format of Principal for a user : arn:aws:iam::${PROJECT_ID}:root) is not supported in RGW.
Does that mean this still works for local RGW users? As in running the script below doesn't raise an AssertionError? I've attached the log output when I run it for comparison, and also the version of the same script I'm using.
import boto3
import logging
logging.basicConfig()
logging.getLogger().setLevel("DEBUG")
iam_client = boto3.client('iam',
aws_access_key_id=<access key of bucket owner (TESTER)>,
aws_secret_access_key=<secret key of bucket owner (TESTER)>
endpoint_url=<IAM_URL>,
region_name='RegionOne'
)
policy_document = '''{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/TESTER1"]},"Action":["sts:AssumeRole"]}]}'''
role_response = iam_client.create_role(
AssumeRolePolicyDocument=policy_document,
Path='/',
RoleName='S3Access',
)
role_policy = '''{"Version":"2012-10-17","Statement":{"Effect":"Allow","Action":"s3:*","Resource":"*"}}'''
response = iam_client.put_role_policy(
RoleName='S3Access',
PolicyName='Policy1',
PolicyDocument=role_policy
)
sts_client = boto3.client('sts',
aws_access_key_id=<access key of bucket accessor (TESTER1)>,
aws_secret_access_key=<secret key of bucket accessor (TESTER1)>
endpoint_url=<STS_URL>,
region_name='RegionOne',
)
response = sts_client.assume_role(
RoleArn=role_response['Role']['Arn'],
RoleSessionName='Bob',
DurationSeconds=3600
)
s3client = boto3.client('s3',
aws_access_key_id = response['Credentials']['AccessKeyId'],
aws_secret_access_key = response['Credentials']['SecretAccessKey'],
aws_session_token = response['Credentials']['SessionToken'],
endpoint_url=<S3_URL>,
region_name='RegionOne',)
# Check we can list objects in a bucket
bucket_name = "my-bucket"
resp = s3client.list_objects_v2(Bucket=bucket_name)
print("Bucket objects:")
for object in resp["Contents"]:
print(object["Key"])
# Test listing all buckets
resp = s3client.list_buckets()
bucket_list = resp["Buckets"]
s3_client_2 = boto3.client('s3',
aws_access_key_id=<access key of bucket owner (TESTER)>,
aws_secret_access_key=<secret key of bucket owner (TESTER)>
endpoint_url=<S3_URL>,
region_name='RegionOne')
resp = s3_client_2.list_buckets()
bucket_list_2 = resp["Buckets"]
print("Checking bucket list comparison")
assert(bucket_list == bucket_list_2)
Updated by Pritha Srivastava over 2 years ago
"Otherwise assuming a role works as expected - for example calling `list_objects_v2` on a bucket lists objects in buckets owned by TESTER, but not buckets owned by TESTER1."
from the above statement I understood that assumerole works fine for local rgw users, or does it work only for listing objects in buckets and not for listing buckets. Please clarify, if it doesnt work I will look into it.
Updated by Sam Mesterton-Gibbons over 2 years ago
Pritha Srivastava wrote:
from the above statement I understood that assumerole works fine for local rgw users, or does it work only for listing objects in buckets and not for listing buckets. Please clarify, if it doesnt work I will look into it.
To clarify: assumerole works for listing objects in buckets, but does not work for listing buckets. However I've only tested that with Openstack Keystone users, not local rgw users. We're currently investigating whether we can test it with local rgw users.
Updated by Sam Mesterton-Gibbons over 2 years ago
Sam Mesterton-Gibbons wrote:
We're currently investigating whether we can test it with local rgw users.
I've now been able to run the same test with a pair of local rgw users, and I can confirm this bug affects local rgw users as well. I can use assumerole to list objects in buckets, but not to list buckets, and the script above raises an AssertionError.