Project

General

Profile

Actions
Copy link

Bug #17872

closed

all swift tests fail with '[Errno 111] Connection refused' because radosgw instance isn't started

Added by Casey Bodley over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

From http://qa-proxy.ceph.com/teuthology/cbodley-2016-11-11_12:15:00-rgw:verify-wip-rgw-compression-type-random---basic-mira/540939/teuthology.log

The radosgw instance is started here at time 18:42:08:

2016-11-11T18:42:08.101 INFO:tasks.rgw.client.0:Restarting daemon
2016-11-11T18:42:08.102 INFO:teuthology.orchestra.run.mira035:Running: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --tool=memcheck radosgw --rgw-frontends 'civetweb port=7280' -n client.0 -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.client.0.stdout 2>&1" 
2016-11-11T18:42:08.106 INFO:tasks.rgw.client.0:Started

Swift tests start almost a minute later, at 18:43:05:

2016-11-11T18:43:05.863 INFO:teuthology.orchestra.run.mira035:Running: "SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw'" 
testAccountHead (test.functional.tests.TestAccount) ... ERROR
testContainerListing (test.functional.tests.TestAccount) ... ERROR
testContainerSerializedInfo (test.functional.tests.TestAccount) ... ERROR
testContainersOrderedByName (test.functional.tests.TestAccount) ... ERROR
testInvalidAuthToken (test.functional.tests.TestAccount) ... ERROR
testInvalidUTF8Path (test.functional.tests.TestAccount) ... ERROR
testLastContainerMarker (test.functional.tests.TestAccount) ... ERROR
testListingLimit (test.functional.tests.TestAccount) ... ERROR
testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ERROR
testPUT (test.functional.tests.TestAccount) ... ERROR
testGetRequest (test.functional.tests.TestAccountNoContainers) ... ERROR
testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ERROR
testAccountHead (test.functional.tests.TestAccountUTF8) ... ERROR

...

======================================================================
ERROR: testAccountHead (test.functional.tests.TestAccount)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/cephtest/swift/test/functional/tests.py", line 104, in setUp
    cls.env.setUp()
  File "/home/ubuntu/cephtest/swift/test/functional/tests.py", line 131, in setUp
    cls.conn.authenticate()
  File "/home/ubuntu/cephtest/swift/test/functional/swift.py", line 131, in authenticate
    connection.request('GET', path, '', headers)
  File "/usr/lib64/python2.7/httplib.py", line 1017, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 826, in send
    self.connect()
  File "/usr/lib64/python2.7/httplib.py", line 807, in connect
    self.timeout, self.source_address)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused

...

The contents of the radosgw log rgw.client.0.log indicate that it didn't actually start until 18:43:09, over 60 seconds after it was invoked (it also doesn't include debug_rgw=20 or debug_ms=1 as configured):

2016-11-11 18:43:09.042131 8c83c80  0 framework: civetweb
2016-11-11 18:43:09.048401 8c83c80  0 framework conf key: port, val: 7280
2016-11-11 18:43:09.152807 8c83c80  0 ceph version 11.0.2-1369-g9768205 (976820570198a98d79affb12b1c9fa5c1bb8d7bf), process memcheck-amd64-, pid 9109
Actions
Copy link

Also available in: Atom PDF