Project

General

Profile

Actions

Bug #10015

closed

rgw sync agent: 403 when syncing object that has tilde in its name

Added by Yehuda Sadeh over 9 years ago. Updated about 9 years ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The cuplrit is the python requests module that the sync agent uses in order to send the http requets. Wittily the module decides that the tilde param should never be escaped and it unescapes it. This collidees with the liburl (that boto usese) escaping scheme that we use before signing the requests. One solution is to modify the requests library. Another solution would be to use different library for sending the requests, maybe something like this:

diff --git a/radosgw_agent/client.py b/radosgw_agent/client.py
index 2b8f027..263bc1f 100644
--- a/radosgw_agent/client.py
+++ b/radosgw_agent/client.py
@@ -5,6 +5,7 @@ import logging
 import random
 import requests
 import urllib
+import string
 from urlparse import urlparse

 from boto.connection import AWSAuthConnection
@@ -111,7 +112,7 @@ def _build_request(conn, method, basepath='', resource = '', headers=None,
         conn, method, path, auth_path, params, headers, data, host)

 def check_result_status(result):
-    if result.status_code / 100 != 2:
+    if result.status / 100 != 2:
         raise code_to_exc.get(result.status_code,
                               HttpError)(result.status_code, result.content)
 def url_safe(component):
@@ -147,16 +148,17 @@ def request(connection, type_, resource, params=None, headers=None,

     request.authorize(connection=connection)

-    handler = getattr(requests, type_)
     boto.log.debug('url = %r\nparams=%r\nheaders=%r\ndata=%r',
                    url, params, request.headers, data)
-    result = handler(url, params=params, headers=request.headers, data=data)
+
+    result = connection.make_request(request.method, request.path, request.body, request.headers)

     check_result_status(result)

     if data or not expect_json:
-        return result.raw
-    return result.json()
+        return result
+
+    return json.loads(result.read())
Actions #1

Updated by Yehuda Sadeh over 9 years ago

  • Assignee changed from Josh Durgin to Alfredo Deza
Actions #2

Updated by Alfredo Deza over 9 years ago

  • Status changed from New to In Progress

Confirmed that requests is doing the unquoting of a quoted url with the tilde char:

In [26]: print requests.get('http://127.0.0.1:8888/?tilde=%7E~').text
{"tilde": "~~"}

That is running a local http server that returns the arguments it receives, I encoded the `~` char and it got 'translated' by `requests`.

Did some investigation and although there are patches available I still believe we should use boto here:

https://github.com/kennethreitz/requests/issues/369

Actions #3

Updated by Sage Weil over 9 years ago

  • Priority changed from Normal to High
Actions #4

Updated by Alfredo Deza over 9 years ago

  • Status changed from In Progress to Fix Under Review
  • Priority changed from High to Normal
Actions #5

Updated by Alfredo Deza over 9 years ago

  • Priority changed from Normal to High
Actions #6

Updated by Yehuda Sadeh over 9 years ago

  • Assignee changed from Alfredo Deza to Josh Durgin
Actions #7

Updated by Yehuda Sadeh over 9 years ago

  • Assignee changed from Josh Durgin to Alfredo Deza
Actions #8

Updated by Alfredo Deza over 9 years ago

  • Status changed from Fix Under Review to Resolved

Changes included in https://github.com/ceph/radosgw-agent/pull/14

merged commit fbcc438 into master

Actions #9

Updated by Ken Dreyer about 9 years ago

Just a note that this fix shipped in the 1.2 release. https://github.com/ceph/radosgw-agent/blob/master/CHANGELOG.rst#12

Actions

Also available in: Atom PDF