Bug #45717
FAIL: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest)
0%
Description
/a/yuriw-2020-05-22_19:55:53-rados-wip-yuri-master_5.22.20-distro-basic-smithi/5083246
2020-05-22T23:15:05.259 INFO:tasks.mgr.dashboard.helper:task (pool/edit, {u'pool_name': u'pool_update_metadata'}) is still executing 2020-05-22T23:15:06.259 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/task?name=pool/edit 2020-05-22T23:15:06.271 INFO:tasks.mgr.dashboard.helper:task (pool/edit, {u'pool_name': u'pool_update_metadata'}) finished 2020-05-22T23:15:11.272 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/pool/pool_update_metadata 2020-05-22T23:15:11.290 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/health/minimal 2020-05-22T23:15:11.386 INFO:tasks.mgr.dashboard.helper:Request PUT to https://smithi074:7791/api/pool/pool_update_metadata 2020-05-22T23:15:13.486 INFO:tasks.mgr.dashboard.helper:task (pool/edit, {u'pool_name': u'pool_update_metadata'}) is still executing 2020-05-22T23:15:14.487 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/task?name=pool/edit 2020-05-22T23:15:14.495 INFO:tasks.mgr.dashboard.helper:task (pool/edit, {u'pool_name': u'pool_update_metadata'}) finished 2020-05-22T23:15:19.496 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/pool/pool_update_metadata 2020-05-22T23:15:19.516 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/health/minimal 2020-05-22T23:15:19.609 INFO:tasks.mgr.dashboard.helper:Request PUT to https://smithi074:7791/api/pool/pool_update_metadata 2020-05-22T23:15:21.708 INFO:tasks.mgr.dashboard.helper:task (pool/edit, {u'pool_name': u'pool_update_metadata'}) is still executing 2020-05-22T23:15:22.709 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/task?name=pool/edit 2020-05-22T23:15:22.717 INFO:tasks.mgr.dashboard.helper:task (pool/edit, {u'pool_name': u'pool_update_metadata'}) finished 2020-05-22T23:15:27.718 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/pool/pool_update_metadata 2020-05-22T23:15:27.739 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi074:7791/api/health/minimal 2020-05-22T23:15:27.837 INFO:tasks.cephfs_test_runner:test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) ... FAIL ... 2020-05-22T23:15:29.048 INFO:tasks.cephfs_test_runner:FAIL: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) 2020-05-22T23:15:29.048 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-05-22T23:15:29.048 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-05-22T23:15:29.048 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri-master_5.22.20/qa/tasks/mgr/dashboard/test_pool.py", line 333, in test_pool_update_metadata 2020-05-22T23:15:29.048 INFO:tasks.cephfs_test_runner: self._validate_pool_properties(properties, self._get_pool(pool_name)) 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri-master_5.22.20/qa/tasks/mgr/dashboard/test_pool.py", line 119, in _validate_pool_properties 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health)) 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner:AssertionError: health={u'status': u'HEALTH_WARN', u'checks': [{u'muted': False, u'detail': [{u'message': u'pg 5.37 is stuck peering since forever, current state peering, last acting [2,0]'}, {u'message': u'pg 6.d is stuck peering since forever, current state peering, last acting [3,1]'}], u'type': u'PG_AVAILABILITY', u'severity': u'HEALTH_WARN', u'summary': {u'count': 4, u'message': u'Reduced data availability: 2 pgs inactive, 2 pgs peering'}}], u'mutes': []} 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner: 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner:Ran 122 tests in 1424.264s 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner: 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner: 2020-05-22T23:15:29.049 INFO:tasks.cephfs_test_runner:====================================================================== 2020-05-22T23:15:29.050 INFO:tasks.cephfs_test_runner:FAIL: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) 2020-05-22T23:15:29.050 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
Related issues
History
#1 Updated by Alfonso MartÃnez 8 months ago
Hi Brad,
I saw that the last ceph-api-nightly-master-backend job runs are successful:
https://jenkins.ceph.com/view/all/job/ceph-api-nightly-master-backend/
So maybe this failure is indicating a negative impact on API/dashboard coming from the tested PR.
It would be great to know the exact PR tested in order to run "jenkins test dashboard backend"
jenkins job and see the result.
#2 Updated by Brad Hubbard 8 months ago
Hi Alfonso,
You can see the branch is "master" in Yuri's test run so this was a baseline run on the HEAD of master at the time (c3321b7686f181e1bcb805a1fb24baced390ae4c). We run these periodically to see what issues pop up.
#3 Updated by Laura Paduano 7 months ago
Also "tasks.mgr.dashboard.test_pool.PoolTest" failures happening in Jenkins now (but the Traceback differs from the original issue, I think I need to create a separate tracker issue):
- https://jenkins.ceph.com/view/mgr-dashboard/job/ceph-api-nightly-master-backend/191/
test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest)
2020-06-07 23:34:49,912.912 INFO:__main__:----------------------------------------------------------------------
2020-06-07 23:34:49,912.912 INFO:__main__:Traceback (most recent call last):
2020-06-07 23:34:49,912.912 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api-nightly-master-backend/qa/tasks/mgr/dashboard/test_pool.py", line 333, in test_pool_update_metadata
2020-06-07 23:34:49,912.912 INFO:__main__: self._validate_pool_properties(properties, self._get_pool(pool_name))
2020-06-07 23:34:49,912.912 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api-nightly-master-backend/qa/tasks/mgr/dashboard/test_pool.py", line 119, in _validate_pool_properties
2020-06-07 23:34:49,912.912 INFO:__main__: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health))
2020-06-07 23:34:49,912.912 INFO:__main__:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK'
2020-06-07 23:34:49,912.912 INFO:__main__:- HEALTH_WARN
2020-06-07 23:34:49,913.913 INFO:__main__:+ HEALTH_OK
2020-06-07 23:34:49,913.913 INFO:__main__: : health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': '1 pool(s) do not have an application enabled', 'count': 1}, 'detail': [{'message': "application not enabled on pool 'pool_update_metadata'"}, {'message': "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications."}], 'muted': False, 'type': 'POOL_APP_NOT_ENABLED'}], 'mutes': []}
2020-06-07 23:34:49,913.913 INFO:__main__:
2020-06-07 23:34:49,913.913 INFO:__main__:----------------------------------------------------------------------
2020-06-07 23:34:49,913.913 INFO:__main__:Ran 122 tests in 2099.146s
2020-06-07 23:34:49,913.913 INFO:__main__:
2020-06-07 23:34:49,914.914 INFO:__main__:
#4 Updated by Sebastian Wagner 7 months ago
https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/2200/
test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) 2020-06-24 08:57:32,473.473 INFO:__main__:---------------------------------------------------------------------- 2020-06-24 08:57:32,473.473 INFO:__main__:Traceback (most recent call last): 2020-06-24 08:57:32,473.473 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 318, in test_pool_update_metadata 2020-06-24 08:57:32,473.473 INFO:__main__: self._validate_pool_properties(props, self._get_pool(pool_name)) 2020-06-24 08:57:32,473.473 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 119, in _validate_pool_properties 2020-06-24 08:57:32,473.473 INFO:__main__: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health)) 2020-06-24 08:57:32,473.473 INFO:__main__:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK' 2020-06-24 08:57:32,474.474 INFO:__main__:- HEALTH_WARN 2020-06-24 08:57:32,474.474 INFO:__main__:+ HEALTH_OK 2020-06-24 08:57:32,474.474 INFO:__main__: : health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': 'Reduced data availability: 1 pg inactive', 'count': 1}, 'detail': [{'message': 'pg 11.14 is stuck inactive for 3m, current state activating, last acting [0,2,3]'}], 'muted': False, 'type': 'PG_AVAILABILITY'}], 'mutes': []} 2020-06-24 08:57:32,474.474 INFO:__main__: 2020-06-24 08:57:32,474.474 INFO:__main__:---------------------------------------------------------------------- 2020-06-24 08:57:32,474.474 INFO:__main__:Ran 122 tests in 1536.051s 2020-06-24 08:57:32,474.474 INFO:__main__: 2020-06-24 08:57:32,475.475 INFO:__main__:
#5 Updated by Sebastian Wagner 7 months ago
https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/1919/
test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) 2020-06-19 16:20:45,941.941 INFO:__main__:---------------------------------------------------------------------- 2020-06-19 16:20:45,941.941 INFO:__main__:Traceback (most recent call last): 2020-06-19 16:20:45,941.941 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 333, in test_pool_update_metadata 2020-06-19 16:20:45,941.941 INFO:__main__: self._validate_pool_properties(properties, self._get_pool(pool_name)) 2020-06-19 16:20:45,941.941 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 119, in _validate_pool_properties 2020-06-19 16:20:45,941.941 INFO:__main__: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health)) 2020-06-19 16:20:45,942.942 INFO:__main__:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK' 2020-06-19 16:20:45,942.942 INFO:__main__:- HEALTH_WARN 2020-06-19 16:20:45,942.942 INFO:__main__:+ HEALTH_OK 2020-06-19 16:20:45,942.942 INFO:__main__: : health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': 'Reduced data availability: 1 pg peering', 'count': 1}, 'detail': [{'message': 'pg 11.13 is stuck peering for 15m, current state peering, last acting [3,0,1]'}], 'muted': False, 'type': 'PG_AVAILABILITY'}], 'mutes': []} 2020-06-19 16:20:45,942.942 INFO:__main__: 2020-06-19 16:20:45,942.942 INFO:__main__:---------------------------------------------------------------------- 2020-06-19 16:20:45,943.943 INFO:__main__:Ran 122 tests in 1674.284s 2020-06-19 16:20:45,943.943 INFO:__main__: 2020-06-19 16:20:45,943.943 INFO:__main__:
#6 Updated by Sebastian Wagner 7 months ago
- Priority changed from Normal to High
#7 Updated by Sebastian Wagner 7 months ago
- Duplicated by Bug #45918: mgr/dashboard: "ceph-api-nightly-master-backend" failure: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) added
#10 Updated by Sebastian Wagner 7 months ago
https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/2267/
test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) 2020-06-24 19:17:03,989.989 INFO:__main__:---------------------------------------------------------------------- 2020-06-24 19:17:03,989.989 INFO:__main__:Traceback (most recent call last): 2020-06-24 19:17:03,989.989 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 333, in test_pool_update_metadata 2020-06-24 19:17:03,990.990 INFO:__main__: self._validate_pool_properties(properties, self._get_pool(pool_name)) 2020-06-24 19:17:03,990.990 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 119, in _validate_pool_properties 2020-06-24 19:17:03,990.990 INFO:__main__: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health)) 2020-06-24 19:17:03,990.990 INFO:__main__:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK' 2020-06-24 19:17:03,990.990 INFO:__main__:- HEALTH_WARN 2020-06-24 19:17:03,990.990 INFO:__main__:+ HEALTH_OK 2020-06-24 19:17:03,990.990 INFO:__main__: : health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': '1 pool(s) do not have an application enabled', 'count': 1}, 'detail': [{'message': "application not enabled on pool 'pool_update_metadata'"}, {'message': "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications."}], 'muted': False, 'type': 'POOL_APP_NOT_ENABLED'}], 'mutes': []} 2020-06-24 19:17:03,991.991 INFO:__main__: 2020-06-24 19:17:03,991.991 INFO:__main__:---------------------------------------------------------------------- 2020-06-24 19:17:03,991.991 INFO:__main__:Ran 122 tests in 2162.395s 2020-06-24 19:17:03,991.991 INFO:__main__: 2020-06-24 19:17:03,991.991 INFO:__main__:
#11 Updated by Kefu Chai 7 months ago
print osd log as well in https://github.com/ceph/ceph/pull/35812 for the root cause of peering.
#12 Updated by Ernesto Puerta 7 months ago
- Status changed from New to In Progress
- Assignee set to Ernesto Puerta
#17 Updated by Alfonso MartÃnez 7 months ago
- Tags changed from nautilus, octpus to nautilus, octopus
#19 Updated by Laura Paduano 6 months ago
- Backport set to nautilus, octopus
#20 Updated by Laura Paduano 6 months ago
- Copied to Backport #46393: nautilus: FAIL: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) added
#21 Updated by Laura Paduano 6 months ago
- Copied to Backport #46394: octopus: FAIL: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) added
#22 Updated by Laura Paduano 6 months ago
Sebastian Wagner wrote:
https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/2200/
[...]
Today's build on a PR: https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/3187/
test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest)
2020-07-08 14:49:55,097.097 INFO:__main__:----------------------------------------------------------------------
2020-07-08 14:49:55,098.098 INFO:__main__:Traceback (most recent call last):
2020-07-08 14:49:55,098.098 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 318, in test_pool_update_metadata
2020-07-08 14:49:55,098.098 INFO:__main__: self._validate_pool_properties(props, self._get_pool(pool_name))
2020-07-08 14:49:55,098.098 INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/dashboard/test_pool.py", line 119, in _validate_pool_properties
2020-07-08 14:49:55,098.098 INFO:__main__: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health))
2020-07-08 14:49:55,098.098 INFO:__main__:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK'
2020-07-08 14:49:55,098.098 INFO:__main__:- HEALTH_WARN
2020-07-08 14:49:55,098.098 INFO:__main__:+ HEALTH_OK
2020-07-08 14:49:55,099.099 INFO:__main__: : health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': 'Reduced data availability: 1 pg inactive, 1 pg peering', 'count': 2}, 'detail': [{'message': 'pg 11.16 is stuck peering for 3m, current state peering, last acting [0,2,1]'}], 'muted': False, 'type': 'PG_AVAILABILITY'}], 'mutes': []}
2020-07-08 14:49:55,099.099 INFO:__main__:
2020-07-08 14:49:55,099.099 INFO:__main__:----------------------------------------------------------------------
2020-07-08 14:49:55,099.099 INFO:__main__:Ran 122 tests in 1536.917s
2020-07-08 14:49:55,099.099 INFO:__main__:
2020-07-08 14:49:55,099.099 INFO:__main__:
#23 Updated by Ernesto Puerta 6 months ago
- Copied to Bug #46573: FAIL: test_pool_update_metadata (tasks.mgr.dashboard.test_pool.PoolTest) added
#24 Updated by Ernesto Puerta 6 months ago
- Category changed from ceph-mgr to API Tests
#25 Updated by Nathan Cutler 6 months ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".