Bug #23536: [rgw] Bucket Policy - HTTP_X_FORWARDED_FOR isn't supported - rgw - Ceph

Actions

Copy link

Bug #23536

closed

[rgw] Bucket Policy - HTTP_X_FORWARDED_FOR isn't supported

Added by Aleksandr Rudenko about 6 years ago. Updated almost 6 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Pritha Srivastava

Target version:

Ceph - v12.2.4

% Done:

Source:

Community (user)

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Luminous 12.2.4

very simple policy:


{
  "Version": "2012-10-17",
  "Id": "S3SimpleIPPolicyTest",
  "Statement": [{
    "Sid": "1",
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": [
      "arn:aws:s3:::enc/*" 
    ],
    "Condition": {
      "IpAddress": {"aws:SourceIp": "178.140.101.160/32"}
    }
  }]
}

if i send request directly to civetweb(not through reverse-proxy) i see in rgw logs:

...
2018-04-01 13:10:48.396523 7f7d984af700 20 HTTP_VERSION=1.1
2018-04-01 13:10:48.396525 7f7d984af700 20 REMOTE_ADDR=178.140.101.160
2018-04-01 13:10:48.396526 7f7d984af700 20 REQUEST_METHOD=GET
2018-04-01 13:10:48.396527 7f7d984af700 20 REQUEST_URI=/enc/comp_my503.png
2018-04-01 13:10:48.396527 7f7d984af700 20 SCRIPT_URI=/enc/comp_my503.png
2018-04-01 13:10:48.396529 7f7d984af700 20 SERVER_PORT=7480
...

all works fine, i succesfully get my object.

But if i send request through reverse-proxy(haproxy in my case) i see in rgw logs:

...
2018-04-01 13:17:16.260797 7f7d96cac700 20 HTTP_X_FORWARDED_FOR=178.140.101.160
2018-04-01 13:17:16.260799 7f7d96cac700 20 REMOTE_ADDR=172.20.35.115
2018-04-01 13:17:16.260800 7f7d96cac700 20 REQUEST_METHOD=GET
2018-04-01 13:17:16.260802 7f7d96cac700 20 REQUEST_URI=/enc/comp_my503.png
2018-04-01 13:17:16.260803 7f7d96cac700 20 SCRIPT_URI=/enc/comp_my503.png
2018-04-01 13:17:16.260804 7f7d96cac700 20 SERVER_PORT=7480
...

and it doesn't work. i get http status code 403.

i see in https://github.com/jgibson/ceph/blob/5f7d9c4ff6c78f65d074dbdf8a181cb9ae09851e/src/rgw/rgw_op.cc#L603-L620 and i don't understand how it works.

And i don't understand hot to use rgw option - rgw_remote_addr_param if i have two or three reverse proxy before civetweb-rgw.

Actions

Copy link

Updated by Matt Benjamin about 6 years ago

Status changed from New to In Progress
Assignee set to Pritha Srivastava

@pritha, could you have a look at this?

Matt

Actions

Copy link

Updated by Pritha Srivastava about 6 years ago

Take a look at https://github.com/ceph/ceph/blob/master/src/common/legacy_config_opts.h#L1350 and https://github.com/ceph/ceph/blob/master/src/common/options.cc#L5246, and then at https://github.com/ceph/ceph/blob/master/src/rgw/rgw_op.cc#L673. I think setting rgw_remote_addr_param to HTTP_X_FORWARDED_FOR in the ceph.conf file should help here. It will help take the originator's ip address as the source ip instead of the ip adress of the proxy server.

Actions

Copy link

Updated by Aleksandr Rudenko about 6 years ago

"172.24.35.115" is internal ip of my reverse-proxy

i set:

"rgw_remote_addr_param": "172.24.35.115"

but it's still not working. I have 403 if i try to get object from public ip specified in policy.

And i have second conceptual question. If i have more then one reverse-proxies? How can i specify them in rgw_remote_addr_param?

i have 4-6 rgw instances and two reverse-proxy in fron of rgw. Customers requests can come from any of my reverse-proxy to rgw.

Actions

Copy link

Updated by Pritha Srivastava about 6 years ago

rgw_remote_addr_param should be set to HTTP_X_FORWARDED_FOR in ceph.conf (and not to any IP address), this will help the originator's ip address to be picked up (instead of the ip address of the reverse proxy)

Actions

Copy link