Bug #48449
Updated by Volker Theile over 3 years ago
This test fails when running locally.
<pre>
2020-12-03 14:17:05,015.015 INFO:tasks.mgr.mgr_test_case:Using port 7789 for dashboard on mgr.x
2020-12-03 14:17:05,015.015 INFO:__main__:> ./bin/ceph config set mgr mgr/dashboard/x/ssl_server_port 7789 --force
2020-12-03 14:17:05,412.412 INFO:tasks.mgr.mgr_test_case:Using port 7790 for dashboard on mgr.y
2020-12-03 14:17:05,413.413 INFO:__main__:> ./bin/ceph config set mgr mgr/dashboard/y/ssl_server_port 7790 --force
2020-12-03 14:17:05,804.804 INFO:tasks.mgr.mgr_test_case:Using port 7791 for dashboard on mgr.z
2020-12-03 14:17:05,804.804 INFO:__main__:> ./bin/ceph config set mgr mgr/dashboard/z/ssl_server_port 7791 --force
2020-12-03 14:17:06,196.196 INFO:__main__:> ps ww -u0
2020-12-03 14:17:06,201.201 INFO:__main__:No match for mgr x
2020-12-03 14:17:06,201.201 INFO:__main__:> ./bin/./ceph-mgr -i x
2020-12-03 14:17:06,287.287 INFO:__main__:> ps ww -u0
2020-12-03 14:17:06,294.294 INFO:__main__:No match for mgr y
2020-12-03 14:17:06,295.295 INFO:__main__:> ./bin/./ceph-mgr -i y
2020-12-03 14:17:06,394.394 INFO:__main__:> ps ww -u0
2020-12-03 14:17:06,404.404 INFO:__main__:No match for mgr z
2020-12-03 14:17:06,407.407 INFO:__main__:> ./bin/./ceph-mgr -i z
2020-12-03 14:17:06,531.531 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:12,302.302 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:12,849.849 INFO:tasks.mgr.mgr_test_case:Available after assign ports (new active y/4780)
2020-12-03 14:17:12,850.850 INFO:__main__:> ./bin/ceph mgr module ls
2020-12-03 14:17:13,401.401 INFO:__main__:> ./bin/ceph dashboard create-self-signed-cert
2020-12-03 14:17:13,925.925 INFO:__main__:> ./bin/ceph config set mgr mgr/dashboard/standby_behaviour error
2020-12-03 14:17:14,421.421 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:14,998.998 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:15,600.600 INFO:tasks.mgr.mgr_test_case:Found dashboard at https://localhost:7790/ (daemon y/4780)
2020-12-03 14:17:15,601.601 INFO:tasks.mgr.test_dashboard:Originally running manager 'y' at https://localhost:7790/
2020-12-03 14:17:15,601.601 INFO:__main__:> ./bin/ceph mgr fail y
2020-12-03 14:17:17,183.183 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:17,871.871 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:18,469.469 INFO:__main__:> ./bin/ceph mgr dump --format=json-pretty
2020-12-03 14:17:19,002.002 INFO:tasks.mgr.mgr_test_case:Found dashboard at https://localhost:7789/ (daemon x/4764)
2020-12-03 14:17:19,003.003 INFO:tasks.mgr.test_dashboard:After failover running manager 'x' at https://localhost:7789/
2020-12-03 14:17:19,006.006 INFO:__main__:> ./bin/ceph config set mgr mgr/dashboard/standby_behaviour redirect
2020-12-03 14:17:19,644.644 INFO:__main__:> ./bin/ceph config set mgr mgr/dashboard/standby_error_status_code 500
2020-12-03 14:17:20,134.134 INFO:__main__:Stopped test: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard) in 19.341034s
2020-12-03 14:17:20,135.135 INFO:__main__:test_standby (tasks.mgr.test_dashboard.TestDashboard) ... ok
2020-12-03 14:17:20,135.135 INFO:__main__:test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard) ... ERROR
2020-12-03 14:17:20,135.135 INFO:__main__:
2020-12-03 14:17:20,135.135 INFO:__main__:======================================================================
2020-12-03 14:17:20,135.135 INFO:__main__:ERROR: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard)
2020-12-03 14:17:20,135.135 INFO:__main__:----------------------------------------------------------------------
2020-12-03 14:17:20,135.135 INFO:__main__:
2020-12-03 14:17:20,136.136 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,136.136 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connection.py", line 158, in _new_conn
2020-12-03 14:17:20,136.136 INFO:__main__: conn = connection.create_connection(
2020-12-03 14:17:20,136.136 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 80, in create_connection
2020-12-03 14:17:20,136.136 INFO:__main__: raise err
2020-12-03 14:17:20,136.136 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 70, in create_connection
2020-12-03 14:17:20,136.136 INFO:__main__: sock.connect(sa)
2020-12-03 14:17:20,136.136 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib64/python3.8/site-packages/gevent/_socket3.py", line 407, in connect
2020-12-03 14:17:20,136.136 INFO:__main__: raise error(err, strerror(err))
2020-12-03 14:17:20,137.137 INFO:__main__:ConnectionRefusedError: [Errno 111] Connection refused
2020-12-03 14:17:20,137.137 INFO:__main__:
2020-12-03 14:17:20,137.137 INFO:__main__:During handling of the above exception, another exception occurred:
2020-12-03 14:17:20,137.137 INFO:__main__:
2020-12-03 14:17:20,137.137 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,137.137 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 597, in urlopen
2020-12-03 14:17:20,137.137 INFO:__main__: httplib_response = self._make_request(conn, method, url,
2020-12-03 14:17:20,138.138 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 343, in _make_request
2020-12-03 14:17:20,138.138 INFO:__main__: self._validate_conn(conn)
2020-12-03 14:17:20,138.138 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
2020-12-03 14:17:20,138.138 INFO:__main__: conn.connect()
2020-12-03 14:17:20,138.138 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connection.py", line 301, in connect
2020-12-03 14:17:20,138.138 INFO:__main__: conn = self._new_conn()
2020-12-03 14:17:20,138.138 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connection.py", line 167, in _new_conn
2020-12-03 14:17:20,138.138 INFO:__main__: raise NewConnectionError(
2020-12-03 14:17:20,139.139 INFO:__main__:urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f34222c1c10>: Failed to establish a new connection: [Errno 111] Connection refused
2020-12-03 14:17:20,139.139 INFO:__main__:
2020-12-03 14:17:20,139.139 INFO:__main__:During handling of the above exception, another exception occurred:
2020-12-03 14:17:20,139.139 INFO:__main__:
2020-12-03 14:17:20,139.139 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,139.139 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
2020-12-03 14:17:20,139.139 INFO:__main__: resp = conn.urlopen(
2020-12-03 14:17:20,139.139 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 637, in urlopen
2020-12-03 14:17:20,139.139 INFO:__main__: retries = retries.increment(method, url, error=e, _pool=self,
2020-12-03 14:17:20,139.139 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 399, in increment
2020-12-03 14:17:20,140.140 INFO:__main__: raise MaxRetryError(_pool, url, error or ResponseError(cause))
2020-12-03 14:17:20,140.140 INFO:__main__:urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=7790): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f34222c1c10>: Failed to establish a new connection: [Errno 111] Connection refused'))
2020-12-03 14:17:20,140.140 INFO:__main__:
2020-12-03 14:17:20,140.140 INFO:__main__:During handling of the above exception, another exception occurred:
2020-12-03 14:17:20,140.140 INFO:__main__:
2020-12-03 14:17:20,140.140 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,140.140 INFO:__main__: File "/ceph/qa/tasks/mgr/test_dashboard.py", line 89, in test_standby_disable_redirect
2020-12-03 14:17:20,140.140 INFO:__main__: r = requests.get(original_uri, allow_redirects=False, verify=False)
2020-12-03 14:17:20,140.140 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/api.py", line 75, in get
2020-12-03 14:17:20,140.140 INFO:__main__: return request('get', url, params=params, **kwargs)
2020-12-03 14:17:20,140.140 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/api.py", line 60, in request
2020-12-03 14:17:20,141.141 INFO:__main__: return session.request(method=method, url=url, **kwargs)
2020-12-03 14:17:20,141.141 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/sessions.py", line 524, in request
2020-12-03 14:17:20,141.141 INFO:__main__: resp = self.send(prep, **send_kwargs)
2020-12-03 14:17:20,141.141 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/sessions.py", line 637, in send
2020-12-03 14:17:20,141.141 INFO:__main__: r = adapter.send(request, **kwargs)
2020-12-03 14:17:20,141.141 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
2020-12-03 14:17:20,141.141 INFO:__main__: raise ConnectionError(e, request=request)
2020-12-03 14:17:20,141.141 INFO:__main__:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=7790): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f34222c1c10>: Failed to establish a new connection: [Errno 111] Connection refused'))
2020-12-03 14:17:20,141.141 INFO:__main__:
2020-12-03 14:17:20,142.142 INFO:__main__:> ip netns list
2020-12-03 14:17:20,145.145 INFO:__main__:> sudo ip link delete ceph-brx
Cannot find device "ceph-brx"
2020-12-03 14:17:20,167.167 INFO:__main__:
2020-12-03 14:17:20,167.167 INFO:__main__:----------------------------------------------------------------------
2020-12-03 14:17:20,167.167 INFO:__main__:Ran 2 tests in 91.041s
2020-12-03 14:17:20,168.168 INFO:__main__:
2020-12-03 14:17:20,168.168 INFO:__main__:FAILED (errors=1)
2020-12-03 14:17:20,168.168 INFO:__main__:
2020-12-03 14:17:20,168.168 INFO:__main__:
2020-12-03 14:17:20,168.168 INFO:__main__:======================================================================
2020-12-03 14:17:20,168.168 INFO:__main__:ERROR: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard)
2020-12-03 14:17:20,169.169 INFO:__main__:----------------------------------------------------------------------
2020-12-03 14:17:20,169.169 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,169.169 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connection.py", line 158, in _new_conn
2020-12-03 14:17:20,169.169 INFO:__main__: conn = connection.create_connection(
2020-12-03 14:17:20,169.169 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 80, in create_connection
2020-12-03 14:17:20,169.169 INFO:__main__: raise err
2020-12-03 14:17:20,169.169 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 70, in create_connection
2020-12-03 14:17:20,169.169 INFO:__main__: sock.connect(sa)
2020-12-03 14:17:20,169.169 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib64/python3.8/site-packages/gevent/_socket3.py", line 407, in connect
2020-12-03 14:17:20,170.170 INFO:__main__: raise error(err, strerror(err))
2020-12-03 14:17:20,170.170 INFO:__main__:ConnectionRefusedError: [Errno 111] Connection refused
2020-12-03 14:17:20,170.170 INFO:__main__:
2020-12-03 14:17:20,170.170 INFO:__main__:During handling of the above exception, another exception occurred:
2020-12-03 14:17:20,170.170 INFO:__main__:
2020-12-03 14:17:20,171.171 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,171.171 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 597, in urlopen
2020-12-03 14:17:20,171.171 INFO:__main__: httplib_response = self._make_request(conn, method, url,
2020-12-03 14:17:20,171.171 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 343, in _make_request
2020-12-03 14:17:20,171.171 INFO:__main__: self._validate_conn(conn)
2020-12-03 14:17:20,172.172 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
2020-12-03 14:17:20,172.172 INFO:__main__: conn.connect()
2020-12-03 14:17:20,172.172 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connection.py", line 301, in connect
2020-12-03 14:17:20,172.172 INFO:__main__: conn = self._new_conn()
2020-12-03 14:17:20,172.172 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connection.py", line 167, in _new_conn
2020-12-03 14:17:20,172.172 INFO:__main__: raise NewConnectionError(
2020-12-03 14:17:20,172.172 INFO:__main__:urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f34222c1c10>: Failed to establish a new connection: [Errno 111] Connection refused
2020-12-03 14:17:20,172.172 INFO:__main__:
2020-12-03 14:17:20,172.172 INFO:__main__:During handling of the above exception, another exception occurred:
2020-12-03 14:17:20,172.172 INFO:__main__:
2020-12-03 14:17:20,173.173 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,173.173 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
2020-12-03 14:17:20,173.173 INFO:__main__: resp = conn.urlopen(
2020-12-03 14:17:20,173.173 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 637, in urlopen
2020-12-03 14:17:20,173.173 INFO:__main__: retries = retries.increment(method, url, error=e, _pool=self,
2020-12-03 14:17:20,173.173 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 399, in increment
2020-12-03 14:17:20,173.173 INFO:__main__: raise MaxRetryError(_pool, url, error or ResponseError(cause))
2020-12-03 14:17:20,173.173 INFO:__main__:urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=7790): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f34222c1c10>: Failed to establish a new connection: [Errno 111] Connection refused'))
2020-12-03 14:17:20,173.173 INFO:__main__:
2020-12-03 14:17:20,173.173 INFO:__main__:During handling of the above exception, another exception occurred:
2020-12-03 14:17:20,174.174 INFO:__main__:
2020-12-03 14:17:20,174.174 INFO:__main__:Traceback (most recent call last):
2020-12-03 14:17:20,174.174 INFO:__main__: File "/ceph/qa/tasks/mgr/test_dashboard.py", line 89, in test_standby_disable_redirect
2020-12-03 14:17:20,174.174 INFO:__main__: r = requests.get(original_uri, allow_redirects=False, verify=False)
2020-12-03 14:17:20,174.174 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/api.py", line 75, in get
2020-12-03 14:17:20,174.174 INFO:__main__: return request('get', url, params=params, **kwargs)
2020-12-03 14:17:20,174.174 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/api.py", line 60, in request
2020-12-03 14:17:20,174.174 INFO:__main__: return session.request(method=method, url=url, **kwargs)
2020-12-03 14:17:20,175.175 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/sessions.py", line 524, in request
2020-12-03 14:17:20,175.175 INFO:__main__: resp = self.send(prep, **send_kwargs)
2020-12-03 14:17:20,175.175 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/sessions.py", line 637, in send
2020-12-03 14:17:20,175.175 INFO:__main__: r = adapter.send(request, **kwargs)
2020-12-03 14:17:20,175.175 INFO:__main__: File "/tmp/tmp.bHswJr3FGl/venv/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
2020-12-03 14:17:20,175.175 INFO:__main__: raise ConnectionError(e, request=request)
2020-12-03 14:17:20,175.175 INFO:__main__:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=7790): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f34222c1c10>: Failed to establish a new connection: [Errno 111] Connection refused'))
</pre>