Bug #46032: tasks.mgr.dashboard.test_ganesha.GaneshaTest: Timed out waiting for daemon mds.a - Dashboard - Ceph

Bug #46032

tasks.mgr.dashboard.test_ganesha.GaneshaTest: Timed out waiting for daemon mds.a

Added by Sebastian Wagner over 3 years ago. Updated about 2 years ago.

Status:

New

Priority:

Normal

Assignee:

Category:

Testing - API

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/1540/

setUpClass (tasks.mgr.dashboard.test_ganesha.GaneshaTest)
2020-06-15 11:26:31,992.992 INFO:__main__:----------------------------------------------------------------------
2020-06-15 11:26:31,993.993 INFO:__main__:Traceback (most recent call last):
2020-06-15 11:26:31,993.993 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_ganesha.py", line 28, in setUpClass
2020-06-15 11:26:31,993.993 INFO:__main__:    super(GaneshaTest, cls).setUpClass()
2020-06-15 11:26:31,994.994 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/helper.py", line 153, in setUpClass
2020-06-15 11:26:31,994.994 INFO:__main__:    cls.mds_cluster.mds_stop()
2020-06-15 11:26:31,994.994 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/cephfs/filesystem.py", line 297, in mds_stop
2020-06-15 11:26:31,995.995 INFO:__main__:    self._one_or_all(mds_id, lambda id_: self.mds_daemons[id_].stop())
2020-06-15 11:26:31,995.995 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/cephfs/filesystem.py", line 273, in _one_or_all
2020-06-15 11:26:31,996.996 INFO:__main__:    p.spawn(cb, mds_id)
2020-06-15 11:26:31,996.996 INFO:__main__:  File "/tmp/tmp.1iEZmcQ52k/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 87, in __exit__
2020-06-15 11:26:31,997.997 INFO:__main__:    for result in self:
2020-06-15 11:26:31,997.997 INFO:__main__:  File "/tmp/tmp.1iEZmcQ52k/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 101, in __next__
2020-06-15 11:26:31,998.998 INFO:__main__:    resurrect_traceback(result)
2020-06-15 11:26:31,998.998 INFO:__main__:  File "/tmp/tmp.1iEZmcQ52k/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 37, in resurrect_traceback
2020-06-15 11:26:31,998.998 INFO:__main__:    reraise(*exc_info)
2020-06-15 11:26:31,999.999 INFO:__main__:  File "/tmp/tmp.1iEZmcQ52k/venv/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-06-15 11:26:31,999.999 INFO:__main__:    raise value
2020-06-15 11:26:31,999.999 INFO:__main__:  File "/tmp/tmp.1iEZmcQ52k/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 24, in capture_traceback
2020-06-15 11:26:32,000.000 INFO:__main__:    return func(*args, **kwargs)
2020-06-15 11:26:32,000.000 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/cephfs/filesystem.py", line 297, in 
2020-06-15 11:26:32,001.001 INFO:__main__:    self._one_or_all(mds_id, lambda id_: self.mds_daemons[id_].stop())
2020-06-15 11:26:32,001.001 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 511, in stop
2020-06-15 11:26:32,001.001 INFO:__main__:    self.daemon_type, self.daemon_id))
2020-06-15 11:26:32,001.001 INFO:__main__:teuthology.exceptions.MaxWhileTries: Timed out waiting for daemon mds.a
2020-06-15 11:26:32,002.002 INFO:__main__:
2020-06-15 11:26:32,002.002 INFO:__main__:----------------------------------------------------------------------
2020-06-15 11:26:32,002.002 INFO:__main__:

Related issues

History

#1 Updated by Sebastian Wagner over 3 years ago

Project changed from Orchestrator to mgr
Category set to 144

#2 Updated by Kefu Chai about 3 years ago

test_get_export (tasks.mgr.dashboard.test_ganesha.GaneshaTest)
2020-07-06 03:28:59,242.242 INFO:__main__:----------------------------------------------------------------------
2020-07-06 03:28:59,242.242 INFO:__main__:Traceback (most recent call last):
2020-07-06 03:28:59,242.242 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_ganesha.py", line 148, in test_get_export
2020-07-06 03:28:59,242.242 INFO:__main__:    self.assertDictEqual(export2, data2)
2020-07-06 03:28:59,243.243 INFO:__main__:AssertionError: {'status': '404 Not Found', 'detail': 'Noth[76 chars]8d2'} != {'detail': 'RGW REST API PUT timed out afte[190 chars]2'}}}
2020-07-06 03:28:59,243.243 INFO:__main__:- {'detail': 'Nothing matches the given URI',
2020-07-06 03:28:59,243.243 INFO:__main__:-  'request_id': '8dc253b6-8467-489b-80e9-11c4c79bd8d2',
2020-07-06 03:28:59,243.243 INFO:__main__:-  'status': '404 Not Found'}
2020-07-06 03:28:59,244.244 INFO:__main__:+ {'component': None,
2020-07-06 03:28:59,244.244 INFO:__main__:+  'detail': 'RGW REST API PUT timed out after 45 seconds '
2020-07-06 03:28:59,244.244 INFO:__main__:+            '(url=http://172.21.1.48:8000/mybucket).',
2020-07-06 03:28:59,244.244 INFO:__main__:+  'status': 500,
2020-07-06 03:28:59,244.244 INFO:__main__:+  'task': {'metadata': {'cluster_id': 'cluster2',
2020-07-06 03:28:59,245.245 INFO:__main__:+                        'fsal': 'RGW',
2020-07-06 03:28:59,245.245 INFO:__main__:+                        'path': 'mybucket'},
2020-07-06 03:28:59,245.245 INFO:__main__:+           'name': 'nfs/create'}}
2020-07-06 03:28:59,245.245 INFO:__main__:
2020-07-06 03:28:59,245.245 INFO:__main__:----------------------------------------------------------------------
2020-07-06 03:28:59,246.246 INFO:__main__:Ran 63 tests in 1313.919s
2020-07-06 03:28:59,246.246 INFO:__main__:

#3 Updated by Kefu Chai about 3 years ago

Related to Bug #45176: octopus: "setUpClass (tasks.mgr.dashboard.test_ganesha.GaneshaTest)" added

#4 Updated by Ernesto Puerta about 3 years ago

Category changed from 144 to 151

#5 Updated by Ernesto Puerta about 3 years ago

Category changed from 151 to API Tests

#6 Updated by Sebastian Wagner about 3 years ago

ERROR: setUpClass (tasks.mgr.dashboard.test_ganesha.GaneshaTest)
2020-07-28 08:13:32,099.099 INFO:__main__:----------------------------------------------------------------------
2020-07-28 08:13:32,099.099 INFO:__main__:Traceback (most recent call last):
2020-07-28 08:13:32,100.100 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_ganesha.py", line 28, in setUpClass
2020-07-28 08:13:32,100.100 INFO:__main__:    super(GaneshaTest, cls).setUpClass()
2020-07-28 08:13:32,100.100 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/helper.py", line 158, in setUpClass
2020-07-28 08:13:32,100.100 INFO:__main__:    cls.mds_cluster.mds_stop()
2020-07-28 08:13:32,101.101 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/cephfs/filesystem.py", line 301, in mds_stop
2020-07-28 08:13:32,102.102 INFO:__main__:    self._one_or_all(mds_id, lambda id_: self.mds_daemons[id_].stop())
2020-07-28 08:13:32,102.102 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/cephfs/filesystem.py", line 277, in _one_or_all
2020-07-28 08:13:32,102.102 INFO:__main__:    p.spawn(cb, mds_id)
2020-07-28 08:13:32,102.102 INFO:__main__:  File "/tmp/tmp.jPjhr3xlki/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 84, in __exit__
2020-07-28 08:13:32,103.103 INFO:__main__:    for result in self:
2020-07-28 08:13:32,103.103 INFO:__main__:  File "/tmp/tmp.jPjhr3xlki/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 98, in __next__
2020-07-28 08:13:32,103.103 INFO:__main__:    resurrect_traceback(result)
2020-07-28 08:13:32,103.103 INFO:__main__:  File "/tmp/tmp.jPjhr3xlki/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 30, in resurrect_traceback
2020-07-28 08:13:32,104.104 INFO:__main__:    raise exc.exc_info[1]
2020-07-28 08:13:32,104.104 INFO:__main__:  File "/tmp/tmp.jPjhr3xlki/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 23, in capture_traceback
2020-07-28 08:13:32,104.104 INFO:__main__:    return func(*args, **kwargs)
2020-07-28 08:13:32,105.105 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/cephfs/filesystem.py", line 301, in 
2020-07-28 08:13:32,105.105 INFO:__main__:    self._one_or_all(mds_id, lambda id_: self.mds_daemons[id_].stop())
2020-07-28 08:13:32,105.105 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 518, in stop
2020-07-28 08:13:32,105.105 INFO:__main__:    self.daemon_type, self.daemon_id))
2020-07-28 08:13:32,106.106 INFO:__main__:teuthology.exceptions.MaxWhileTries: Timed out waiting for daemon mds.a
2020-07-28 08:13:32,106.106 INFO:__main__:
2020-07-28 08:13:32,106.106 INFO:__main__:----------------------------------------------------------------------
2020-07-28 08:13:32,107.107 INFO:__main__:Ran 62 tests in 975.271s
2020-07-28 08:13:32,107.107 INFO:__main__:
2020-07-28 08:13:32,107.107 INFO:__main__:

https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/4495/

#7 Updated by Josh Durgin about 3 years ago

https://jenkins.ceph.com/job/ceph-api/227/

2020-08-05 23:15:20,201.201 INFO:__main__:----------------------------------------------------------------------
2020-08-05 23:15:20,201.201 INFO:__main__:Traceback (most recent call last):
2020-08-05 23:15:20,202.202 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/test_ganesha.py", line 28, in setUpClass
2020-08-05 23:15:20,202.202 INFO:__main__:    super(GaneshaTest, cls).setUpClass()
2020-08-05 23:15:20,203.203 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/helper.py", line 158, in setUpClass
2020-08-05 23:15:20,203.203 INFO:__main__:    cls.mds_cluster.mds_stop()
2020-08-05 23:15:20,203.203 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/cephfs/filesystem.py", line 301, in mds_stop
2020-08-05 23:15:20,203.203 INFO:__main__:    self._one_or_all(mds_id, lambda id_: self.mds_daemons[id_].stop())
2020-08-05 23:15:20,204.204 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/cephfs/filesystem.py", line 277, in _one_or_all
2020-08-05 23:15:20,204.204 INFO:__main__:    p.spawn(cb, mds_id)
2020-08-05 23:15:20,205.205 INFO:__main__:  File "/tmp/tmp.D2dMxEnObe/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 84, in __exit__
2020-08-05 23:15:20,205.205 INFO:__main__:    for result in self:
2020-08-05 23:15:20,205.205 INFO:__main__:  File "/tmp/tmp.D2dMxEnObe/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 98, in __next__
2020-08-05 23:15:20,206.206 INFO:__main__:    resurrect_traceback(result)
2020-08-05 23:15:20,206.206 INFO:__main__:  File "/tmp/tmp.D2dMxEnObe/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 30, in resurrect_traceback
2020-08-05 23:15:20,206.206 INFO:__main__:    raise exc.exc_info[1]
2020-08-05 23:15:20,207.207 INFO:__main__:  File "/tmp/tmp.D2dMxEnObe/venv/lib/python3.6/site-packages/teuthology/parallel.py", line 23, in capture_traceback
2020-08-05 23:15:20,207.207 INFO:__main__:    return func(*args, **kwargs)
2020-08-05 23:15:20,207.207 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/cephfs/filesystem.py", line 301, in 
2020-08-05 23:15:20,207.207 INFO:__main__:    self._one_or_all(mds_id, lambda id_: self.mds_daemons[id_].stop())
2020-08-05 23:15:20,208.208 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 519, in stop
2020-08-05 23:15:20,208.208 INFO:__main__:    self.daemon_type, self.daemon_id))
2020-08-05 23:15:20,209.209 INFO:__main__:teuthology.exceptions.MaxWhileTries: Timed out waiting for daemon mds.a
2020-08-05 23:15:20,209.209 INFO:__main__:
2020-08-05 23:15:20,209.209 INFO:__main__:----------------------------------------------------------------------
2020-08-05 23:15:20,210.210 INFO:__main__:Ran 62 tests in 1008.729s
2020-08-05 23:15:20,210.210 INFO:__main__:
2020-08-05 23:15:20,211.211 INFO:__main__: