Project

General

Profile

Bug #48766

qa: Test failure: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient)

Added by Jos Collin 3 months ago. Updated about 2 months ago.

Status:
Duplicate
Priority:
Normal
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
pacific,octopus,nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
mgr/volumes
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

test_evict_client fails in [1] and [2].

[1] http://qa-proxy.ceph.com/teuthology/jcollin-2021-01-05_16:12:23-fs-wip-jcollin-dispatch-throttle-new-config-values-distro-basic-smithi/5756417/teuthology.log
[2] http://qa-proxy.ceph.com/teuthology/jcollin-2021-01-05_16:14:34-fs-wip-jcollin-dispatch-throttle-default-config-values-distro-basic-smithi/5756519/teuthology.log

2021-01-05T16:34:28.944 INFO:tasks.cephfs_test_runner:test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient) ... ERROR
2021-01-05T16:34:28.945 INFO:tasks.cephfs_test_runner:
2021-01-05T16:34:28.945 INFO:tasks.cephfs_test_runner:======================================================================
2021-01-05T16:34:28.945 INFO:tasks.cephfs_test_runner:ERROR: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient)
2021-01-05T16:34:28.946 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2021-01-05T16:34:28.946 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2021-01-05T16:34:28.946 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs/test_volume_client.py", line 461, in test_evict_client
2021-01-05T16:34:28.947 INFO:tasks.cephfs_test_runner:    guest_entity=guest_entity
2021-01-05T16:34:28.947 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs/test_volume_client.py", line 41, in _volume_client_python
2021-01-05T16:34:28.947 INFO:tasks.cephfs_test_runner:    vol_prefix=vol_prefix, ns_prefix=ns_prefix))
2021-01-05T16:34:28.947 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs/mount.py", line 667, in run_python
2021-01-05T16:34:28.948 INFO:tasks.cephfs_test_runner:    p.wait()
2021-01-05T16:34:28.948 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait
2021-01-05T16:34:28.948 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2021-01-05T16:34:28.949 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 183, in _raise_for_status
2021-01-05T16:34:28.949 INFO:tasks.cephfs_test_runner:    node=self.hostname, label=self.label
2021-01-05T16:34:28.949 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi062 with status 1: 'sudo adjust-ulimits daemon-helper kill python3 -c \'\nfrom __future__ import print_function\nfrom ceph_volume_client import CephFSVolumeClient, VolumePath\nfrom sys import version_info as sys_version_info\nfrom rados import OSError as rados_OSError\nimport logging\nlog = logging.getLogger("ceph_volume_client")\nlog.addHandler(logging.StreamHandler())\nlog.setLevel(logging.DEBUG)\nvc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None)\nvc.connect()\n\nvp = VolumePath("grpid", "volid_0")\nvc.deauthorize(vp, "guest")\nvc.evict("guest", volume_path=vp)\n\nvc.disconnect()\n        \''
2021-01-05T16:34:28.950 INFO:tasks.cephfs_test_runner:
2021-01-05T16:34:28.950 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2021-01-05T16:34:28.950 INFO:tasks.cephfs_test_runner:Ran 9 tests in 403.753s
2021-01-05T16:34:28.950 INFO:tasks.cephfs_test_runner:
2021-01-05T16:34:28.951 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
2021-01-05T16:34:28.951 INFO:tasks.cephfs_test_runner:
2021-01-05T16:34:28.951 INFO:tasks.cephfs_test_runner:======================================================================
2021-01-05T16:34:28.952 INFO:tasks.cephfs_test_runner:ERROR: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient)
2021-01-05T16:34:28.952 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2021-01-05T16:34:28.952 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2021-01-05T16:34:28.953 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs/test_volume_client.py", line 461, in test_evict_client
2021-01-05T16:34:28.953 INFO:tasks.cephfs_test_runner:    guest_entity=guest_entity
2021-01-05T16:34:28.953 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs/test_volume_client.py", line 41, in _volume_client_python
2021-01-05T16:34:28.953 INFO:tasks.cephfs_test_runner:    vol_prefix=vol_prefix, ns_prefix=ns_prefix))
2021-01-05T16:34:28.954 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs/mount.py", line 667, in run_python
2021-01-05T16:34:28.954 INFO:tasks.cephfs_test_runner:    p.wait()
2021-01-05T16:34:28.954 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait
2021-01-05T16:34:28.955 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2021-01-05T16:34:28.955 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 183, in _raise_for_status
2021-01-05T16:34:28.956 INFO:tasks.cephfs_test_runner:    node=self.hostname, label=self.label
2021-01-05T16:34:28.956 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi062 with status 1: 'sudo adjust-ulimits daemon-helper kill python3 -c \'\nfrom __future__ import print_function\nfrom ceph_volume_client import CephFSVolumeClient, VolumePath\nfrom sys import version_info as sys_version_info\nfrom rados import OSError as rados_OSError\nimport logging\nlog = logging.getLogger("ceph_volume_client")\nlog.addHandler(logging.StreamHandler())\nlog.setLevel(logging.DEBUG)\nvc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None)\nvc.connect()\n\nvp = VolumePath("grpid", "volid_0")\nvc.deauthorize(vp, "guest")\nvc.evict("guest", volume_path=vp)\n\nvc.disconnect()\n        \''
2021-01-05T16:34:28.957 INFO:tasks.cephfs_test_runner:
2021-01-05T16:34:28.957 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 94, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs_test_runner.py", line 211, in task
    raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient)
2021-01-05T16:34:28.977 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=574ab89584944528ad36b6d572b39d72
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 94, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-jcollin-dispatch-throttle-new-config-values/qa/tasks/cephfs_test_runner.py", line 211, in task
    raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient)

History

#1 Updated by Patrick Donnelly 3 months ago

master failure: /ceph/teuthology-archive/teuthology-2020-12-27_03:15:03-fs-master-distro-basic-smithi/5738903/teuthology.log

#2 Updated by Patrick Donnelly 3 months ago

  • Status changed from New to Triaged
  • Assignee set to Kotresh Hiremath Ravishankar
  • Target version set to v16.0.0
  • Source set to Q/A
  • Component(FS) mgr/volumes added
  • Labels (FS) qa-failure added

#3 Updated by Patrick Donnelly 3 months ago

  • Target version changed from v16.0.0 to v17.0.0
  • Backport set to pacific,octopus,nautilus

#4 Updated by Kotresh Hiremath Ravishankar 3 months ago

The issue is no longer applicable to master as the test is removed as part of removing ceph_volume_client [1]

From the below errors from log, this fix for [2] should fix this as well.

-------
2021-01-05T17:44:13.775 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to evict client with auth_name=guest, client_metadata.root=/volumes/grpid/volid_0 from mds 0/8889: args must be encodeable as a bytearray
2021-01-05T17:44:13.776 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last):
2021-01-05T17:44:13.776 INFO:teuthology.orchestra.run.smithi129.stderr: File "<string>", line 15, in <module>
2021-01-05T17:44:13.776 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib/python3.6/site-packages/ceph_volume_client.py", line 462, in evict
2021-01-05T17:44:13.776 INFO:teuthology.orchestra.run.smithi129.stderr: raise EvictionError(msg)
2021-01-05T17:44:13.776 INFO:teuthology.orchestra.run.smithi129.stderr:ceph_volume_client.EvictionError: Failed to evict client with auth_name=guest, client_metadata.root=/volumes/grpid/volid_0 from mds 0/8889: args must be encodeable as a bytearray
------------

[1] https://github.com/ceph/ceph/pull/38846
[2] https://tracker.ceph.com/issues/48830

#5 Updated by Kotresh Hiremath Ravishankar 3 months ago

  • Status changed from Triaged to In Progress

#6 Updated by Kotresh Hiremath Ravishankar about 2 months ago

  • Status changed from In Progress to Duplicate

Also available in: Atom PDF