Bug #45528
mgr/dashboard:ceph-api-nautilus-backend AssertionError "test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard)"
Status:
New
Priority:
Normal
Assignee:
-
Category:
Testing - API
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2020-05-13 08:06:42,324.324 INFO:__main__:FAIL: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard)
2020-05-13 08:06:42,324.324 INFO:__main__:----------------------------------------------------------------------
2020-05-13 08:06:42,324.324 INFO:__main__:Traceback (most recent call last):
2020-05-13 08:06:42,324.324 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api-nightly-nautilus-backend/qa/tasks/mgr/test_dashboard.py", line 98, in test_standby_disable_redirect
2020-05-13 08:06:42,325.325 INFO:__main__: self.assertEqual(r.status_code, 503)
2020-05-13 08:06:42,325.325 INFO:__main__:AssertionError: 500 != 503
2020-05-13 08:06:42,325.325 INFO:__main__:
2020-05-13 08:06:42,325.325 INFO:__main__:----------------------------------------------------------------------
2020-05-13 08:06:42,325.325 INFO:__main__:Ran 2 tests in 84.876s
2020-05-13 08:06:42,326.326 INFO:__main__:
2020-05-13 08:06:42,326.326 INFO:__main__:FAILED (failures=1)
2020-05-13 08:06:42,326.326 INFO:__main__:
2020-05-13 08:06:42,326.326 INFO:__main__:======================================================================
2020-05-13 08:06:42,327.327 INFO:__main__:FAIL: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard)
2020-05-13 08:06:42,327.327 INFO:__main__:----------------------------------------------------------------------
2020-05-13 08:06:42,327.327 INFO:__main__:Traceback (most recent call last):
2020-05-13 08:06:42,327.327 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api-nightly-nautilus-backend/qa/tasks/mgr/test_dashboard.py", line 98, in test_standby_disable_redirect
2020-05-13 08:06:42,328.328 INFO:__main__: self.assertEqual(r.status_code, 503)
2020-05-13 08:06:42,328.328 INFO:__main__:AssertionError: 500 != 503
- https://jenkins.ceph.com/view/all/job/ceph-api-nightly-nautilus-backend/163
- https://jenkins.ceph.com/view/mgr-dashboard/job/ceph-api-nightly-nautilus-backend/312/console
History
#1 Updated by Lenz Grimmer over 3 years ago
Adding some more context from the log - before the test, the setting standby_error_status_code is explicitly set to 500:
2020-05-13 08:06:28,107.107 INFO:tasks.mgr.mgr_test_case:Found dashboard at https://slave-ubuntu10.front.sepia.ceph.com:7790/ (daemon x/4768) 2020-05-13 08:06:28,107.107 INFO:tasks.mgr.test_dashboard:After failover running manager 'x' at https://slave-ubuntu10.front.sepia.ceph.com:7790/ /tmp/tmp.VCfxkcGuTh/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-05-13 08:06:28,128.128 INFO:__main__:Running ['./bin/ceph', 'config', 'set', 'mgr', 'mgr/dashboard/standby_error_status_code', '503'] /tmp/tmp.VCfxkcGuTh/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-05-13 08:06:33,412.412 INFO:__main__:test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard) ... FAIL 2020-05-13 08:06:33,412.412 INFO:__main__:Running ['./bin/ceph', 'config', 'set', 'mgr', 'mgr/dashboard/standby_behaviour', 'redirect'] 2020-05-13 08:06:41,783.783 INFO:__main__:Running ['./bin/ceph', 'config', 'set', 'mgr', 'mgr/dashboard/standby_error_status_code', '500'] 2020-05-13 08:06:42,322.322 INFO:__main__:Stopped test: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard) in 38.240224s 2020-05-13 08:06:42,323.323 INFO:__main__: 2020-05-13 08:06:42,324.324 INFO:__main__:====================================================================== 2020-05-13 08:06:42,324.324 INFO:__main__:FAIL: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard) 2020-05-13 08:06:42,324.324 INFO:__main__:---------------------------------------------------------------------- 2020-05-13 08:06:42,324.324 INFO:__main__:Traceback (most recent call last): 2020-05-13 08:06:42,324.324 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api-nightly-nautilus-backend/qa/tasks/mgr/test_dashboard.py", line 98, in test_standby_disable_redirect 2020-05-13 08:06:42,325.325 INFO:__main__: self.assertEqual(r.status_code, 503) 2020-05-13 08:06:42,325.325 INFO:__main__:AssertionError: 500 != 503 2020-05-13 08:06:42,325.325 INFO:__main__: 2020-05-13 08:06:42,325.325 INFO:__main__:---------------------------------------------------------------------- 2020-05-13 08:06:42,325.325 INFO:__main__:Ran 2 tests in 84.876s 2020-05-13 08:06:42,326.326 INFO:__main__: 2020-05-13 08:06:42,326.326 INFO:__main__:FAILED (failures=1) 2020-05-13 08:06:42,326.326 INFO:__main__: 2020-05-13 08:06:42,326.326 INFO:__main__:====================================================================== 2020-05-13 08:06:42,327.327 INFO:__main__:FAIL: test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard) 2020-05-13 08:06:42,327.327 INFO:__main__:---------------------------------------------------------------------- 2020-05-13 08:06:42,327.327 INFO:__main__:Traceback (most recent call last): 2020-05-13 08:06:42,327.327 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api-nightly-nautilus-backend/qa/tasks/mgr/test_dashboard.py", line 98, in test_standby_disable_redirect 2020-05-13 08:06:42,328.328 INFO:__main__: self.assertEqual(r.status_code, 503) 2020-05-13 08:06:42,328.328 INFO:__main__:AssertionError: 500 != 503 2020-05-13 08:06:42,328.328 INFO:__main__:
#2 Updated by Sebastian Wagner over 3 years ago
https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/2323/
test_standby_disable_redirect (tasks.mgr.test_dashboard.TestDashboard)
2020-06-25 11:34:46,855.855 INFO:__main__:----------------------------------------------------------------------
2020-06-25 11:34:46,856.856 INFO:__main__:Traceback (most recent call last):
2020-06-25 11:34:46,856.856 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
2020-06-25 11:34:46,856.856 INFO:__main__: (self._dns_host, self.port), self.timeout, **extra_kw)
2020-06-25 11:34:46,856.856 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
2020-06-25 11:34:46,856.856 INFO:__main__: raise err
2020-06-25 11:34:46,856.856 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
2020-06-25 11:34:46,856.856 INFO:__main__: sock.connect(sa)
2020-06-25 11:34:46,856.856 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/gevent/_socket3.py", line 407, in connect
2020-06-25 11:34:46,856.856 INFO:__main__: raise error(err, strerror(err))
2020-06-25 11:34:46,856.856 INFO:__main__:ConnectionRefusedError: [Errno 111] Connection refused
2020-06-25 11:34:46,856.856 INFO:__main__:
2020-06-25 11:34:46,857.857 INFO:__main__:During handling of the above exception, another exception occurred:
2020-06-25 11:34:46,857.857 INFO:__main__:
2020-06-25 11:34:46,857.857 INFO:__main__:Traceback (most recent call last):
2020-06-25 11:34:46,857.857 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
2020-06-25 11:34:46,857.857 INFO:__main__: chunked=chunked)
2020-06-25 11:34:46,857.857 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
2020-06-25 11:34:46,857.857 INFO:__main__: self._validate_conn(conn)
2020-06-25 11:34:46,857.857 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
2020-06-25 11:34:46,857.857 INFO:__main__: conn.connect()
2020-06-25 11:34:46,857.857 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connection.py", line 301, in connect
2020-06-25 11:34:46,857.857 INFO:__main__: conn = self._new_conn()
2020-06-25 11:34:46,858.858 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
2020-06-25 11:34:46,858.858 INFO:__main__: self, "Failed to establish a new connection: %s" % e)
2020-06-25 11:34:46,858.858 INFO:__main__:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused
2020-06-25 11:34:46,858.858 INFO:__main__:
2020-06-25 11:34:46,858.858 INFO:__main__:During handling of the above exception, another exception occurred:
2020-06-25 11:34:46,858.858 INFO:__main__:
2020-06-25 11:34:46,858.858 INFO:__main__:Traceback (most recent call last):
2020-06-25 11:34:46,858.858 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
2020-06-25 11:34:46,858.858 INFO:__main__: timeout=timeout
2020-06-25 11:34:46,858.858 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
2020-06-25 11:34:46,858.858 INFO:__main__: _stacktrace=sys.exc_info()[2])
2020-06-25 11:34:46,859.859 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
2020-06-25 11:34:46,859.859 INFO:__main__: raise MaxRetryError(_pool, url, error or ResponseError(cause))
2020-06-25 11:34:46,859.859 INFO:__main__:urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bionic-huge--4f0a9205-ec46-4117-955c-7cb614accf17', port=7789): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
2020-06-25 11:34:46,859.859 INFO:__main__:
2020-06-25 11:34:46,859.859 INFO:__main__:During handling of the above exception, another exception occurred:
2020-06-25 11:34:46,859.859 INFO:__main__:
2020-06-25 11:34:46,859.859 INFO:__main__:Traceback (most recent call last):
2020-06-25 11:34:46,859.859 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_dashboard.py", line 87, in test_standby_disable_redirect
2020-06-25 11:34:46,859.859 INFO:__main__: r = requests.get(original_uri, allow_redirects=False, verify=False)
2020-06-25 11:34:46,859.859 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/requests/api.py", line 75, in get
2020-06-25 11:34:46,860.860 INFO:__main__: return request('get', url, params=params, **kwargs)
2020-06-25 11:34:46,860.860 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/requests/api.py", line 60, in request
2020-06-25 11:34:46,860.860 INFO:__main__: return session.request(method=method, url=url, **kwargs)
2020-06-25 11:34:46,860.860 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/requests/sessions.py", line 524, in request
2020-06-25 11:34:46,860.860 INFO:__main__: resp = self.send(prep, **send_kwargs)
2020-06-25 11:34:46,860.860 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/requests/sessions.py", line 637, in send
2020-06-25 11:34:46,860.860 INFO:__main__: r = adapter.send(request, **kwargs)
2020-06-25 11:34:46,860.860 INFO:__main__: File "/tmp/tmp.TIrUUSSykC/venv/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
2020-06-25 11:34:46,860.860 INFO:__main__: raise ConnectionError(e, request=request)
2020-06-25 11:34:46,860.860 INFO:__main__:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='bionic-huge--4f0a9205-ec46-4117-955c-7cb614accf17', port=7789): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
2020-06-25 11:34:46,860.860 INFO:__main__:
2020-06-25 11:34:46,861.861 INFO:__main__:----------------------------------------------------------------------
2020-06-25 11:34:46,861.861 INFO:__main__:
#3 Updated by Laura Paduano about 3 years ago
- Category changed from 151 to API Tests
#4 Updated by Kefu Chai about 3 years ago
2020-07-21 09:46:02,553.553 INFO:__main__:----------------------------------------------------------------------
2020-07-21 09:46:02,554.554 INFO:__main__:Traceback (most recent call last):
2020-07-21 09:46:02,554.554 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
2020-07-21 09:46:02,554.554 INFO:__main__: (self._dns_host, self.port), self.timeout, **extra_kw)
2020-07-21 09:46:02,554.554 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
2020-07-21 09:46:02,554.554 INFO:__main__: raise err
2020-07-21 09:46:02,554.554 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
2020-07-21 09:46:02,554.554 INFO:__main__: sock.connect(sa)
2020-07-21 09:46:02,554.554 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/gevent/_socket3.py", line 407, in connect
2020-07-21 09:46:02,555.555 INFO:__main__: raise error(err, strerror(err))
2020-07-21 09:46:02,555.555 INFO:__main__:ConnectionRefusedError: [Errno 111] Connection refused
2020-07-21 09:46:02,555.555 INFO:__main__:
2020-07-21 09:46:02,555.555 INFO:__main__:During handling of the above exception, another exception occurred:
2020-07-21 09:46:02,555.555 INFO:__main__:
2020-07-21 09:46:02,555.555 INFO:__main__:Traceback (most recent call last):
2020-07-21 09:46:02,555.555 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
2020-07-21 09:46:02,556.556 INFO:__main__: chunked=chunked)
2020-07-21 09:46:02,556.556 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
2020-07-21 09:46:02,556.556 INFO:__main__: self._validate_conn(conn)
2020-07-21 09:46:02,556.556 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
2020-07-21 09:46:02,556.556 INFO:__main__: conn.connect()
2020-07-21 09:46:02,556.556 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connection.py", line 301, in connect
2020-07-21 09:46:02,556.556 INFO:__main__: conn = self._new_conn()
2020-07-21 09:46:02,557.557 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
2020-07-21 09:46:02,557.557 INFO:__main__: self, "Failed to establish a new connection: %s" % e)
2020-07-21 09:46:02,557.557 INFO:__main__:urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused
2020-07-21 09:46:02,557.557 INFO:__main__:
2020-07-21 09:46:02,557.557 INFO:__main__:During handling of the above exception, another exception occurred:
2020-07-21 09:46:02,557.557 INFO:__main__:
2020-07-21 09:46:02,557.557 INFO:__main__:Traceback (most recent call last):
2020-07-21 09:46:02,558.558 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
2020-07-21 09:46:02,558.558 INFO:__main__: timeout=timeout
2020-07-21 09:46:02,558.558 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
2020-07-21 09:46:02,558.558 INFO:__main__: _stacktrace=sys.exc_info()[2])
2020-07-21 09:46:02,558.558 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
2020-07-21 09:46:02,558.558 INFO:__main__: raise MaxRetryError(_pool, url, error or ResponseError(cause))
2020-07-21 09:46:02,558.558 INFO:__main__:urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='slave-ubuntu11.front.sepia.ceph.com', port=7789): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
2020-07-21 09:46:02,559.559 INFO:__main__:
2020-07-21 09:46:02,559.559 INFO:__main__:During handling of the above exception, another exception occurred:
2020-07-21 09:46:02,559.559 INFO:__main__:
2020-07-21 09:46:02,559.559 INFO:__main__:Traceback (most recent call last):
2020-07-21 09:46:02,559.559 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_dashboard.py", line 87, in test_standby_disable_redirect
2020-07-21 09:46:02,559.559 INFO:__main__: r = requests.get(original_uri, allow_redirects=False, verify=False)
2020-07-21 09:46:02,559.559 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/requests/api.py", line 75, in get
2020-07-21 09:46:02,560.560 INFO:__main__: return request('get', url, params=params, **kwargs)
2020-07-21 09:46:02,560.560 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/requests/api.py", line 60, in request
2020-07-21 09:46:02,560.560 INFO:__main__: return session.request(method=method, url=url, **kwargs)
2020-07-21 09:46:02,560.560 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/requests/sessions.py", line 524, in request
2020-07-21 09:46:02,560.560 INFO:__main__: resp = self.send(prep, **send_kwargs)
2020-07-21 09:46:02,560.560 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/requests/sessions.py", line 637, in send
2020-07-21 09:46:02,560.560 INFO:__main__: r = adapter.send(request, **kwargs)
2020-07-21 09:46:02,560.560 INFO:__main__: File "/tmp/tmp.xyJc9GEYKC/venv/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
2020-07-21 09:46:02,561.561 INFO:__main__: raise ConnectionError(e, request=request)
2020-07-21 09:46:02,561.561 INFO:__main__:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='slave-ubuntu11.front.sepia.ceph.com', port=7789): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
2020-07-21 09:46:02,561.561 INFO:__main__:
2020-07-21 09:46:02,561.561 INFO:__main__:----------------------------------------------------------------------
2020-07-21 09:46:02,561.561 INFO:__main__:Ran 2 tests in 93.358s
2020-07-21 09:46:02,561.561 INFO:__main__:
2020-07-21 09:46:02,562.562 INFO:__main__:
https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/4062/
#5 Updated by Laura Paduano about 3 years ago
- Description updated (diff)
#6 Updated by Brad Hubbard almost 3 years ago
Still seeing this, https://jenkins.ceph.com/job/ceph-api/7700/
#7 Updated by Ernesto Puerta over 2 years ago
- Project changed from mgr to Dashboard
- Category changed from API Tests to Testing - API