Project

General

Profile

Bug #44437

qa:test_config_session_timeout failed with incorrect options

Added by Xiubo Li 27 days ago. Updated 4 days ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
fs
Component(FS):
qa-suite
Labels (FS):
qa
Pull request ID:
Crash signature:

Description

2020-03-05 04:10:04,311.311 INFO:__main__:Running ['./bin/ceph', 'daemon', 'mds.a', 'client', 'config', '9430', '120']
...
2020-03-05 04:10:06,178.178 INFO:__main__:
2020-03-05 04:10:06,178.178 INFO:__main__:======================================================================
2020-03-05 04:10:06,178.178 INFO:__main__:ERROR: test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery)
2020-03-05 04:10:06,179.179 INFO:__main__:----------------------------------------------------------------------
2020-03-05 04:10:06,179.179 INFO:__main__:Traceback (most recent call last):
2020-03-05 04:10:06,179.179 INFO:__main__: File "/data/ceph/qa/tasks/cephfs/test_client_recovery.py", line 621, in test_config_session_timeout
2020-03-05 04:10:06,180.180 INFO:__main__: self.fs.mds_asok(['client', 'config', '%s' % mount_a_gid, 'timeout', '%s' % (session_timeout * 2)])
2020-03-05 04:10:06,180.180 INFO:__main__: File "/data/ceph/qa/tasks/cephfs/filesystem.py", line 1001, in mds_asok
2020-03-05 04:10:06,180.180 INFO:__main__: return self.json_asok(command, 'mds', mds_id, timeout=timeout)
2020-03-05 04:10:06,180.180 INFO:__main__: File "/data/ceph/qa/tasks/cephfs/filesystem.py", line 227, in json_asok
2020-03-05 04:10:06,180.180 INFO:__main__: return json.loads(response_data)
2020-03-05 04:10:06,181.181 INFO:__main__: File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
2020-03-05 04:10:06,181.181 INFO:__main__: return default_decoder.decode(s)
2020-03-05 04:10:06,181.181 INFO:
_main__: File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
2020-03-05 04:10:06,181.181 INFO:__main__: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2020-03-05 04:10:06,182.182 INFO:__main__: File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
2020-03-05 04:10:06,182.182 INFO:__main__: raise ValueError("No JSON object could be decoded")
2020-03-05 04:10:06,182.182 INFO:__main__:ValueError: No JSON object could be decoded
2020-03-05 04:10:06,182.182 INFO:__main__:

The 'timeout' is removed somewhere before calling the remote.run()


Related issues

Copied to fs - Backport #44844: octopus: qa:test_config_session_timeout failed with incorrect options New

History

#1 Updated by Xiubo Li 27 days ago

  • Status changed from New to In Progress

#2 Updated by Patrick Donnelly 27 days ago

  • Target version set to v16.0.0

#3 Updated by Xiubo Li 26 days ago

More detail logs:


# python2 ../qa/tasks/vstart_runner.py tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout

...

2020-03-05 21:03:28,866.866 INFO:__main__:Running ['./bin/ceph', 'fs', 'dump', '--format=json']
2020-03-05 21:03:29,370.370 INFO:__main__:Running ['./bin/ceph', 'daemon', 'mds.b', 'session', 'config', '20296', '120']
2020-03-05 21:03:29,635.635 INFO:tasks.cephfs.filesystem:_json_asok output: ERROR: (22) Invalid argument
Invalid config option: 120

2020-03-05 21:03:29,648.648 INFO:__main__:test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ERROR
2020-03-05 21:03:29,649.649 INFO:__main__:Running ['stat', '--file-system', '--printf=%T\n', '--', '/tmp/tmp7talcY/mnt.0']
2020-03-05 21:03:29,661.661 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/tmp7talcY/mnt.0
2020-03-05 21:03:29,661.661 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on local...
2020-03-05 21:03:29,661.661 INFO:__main__:Running ['sudo', 'fusermount', '-u', '/tmp/tmp7talcY/mnt.0']
2020-03-05 21:03:29,715.715 INFO:__main__:Running ['stat', '--file-system', '--printf=%T\n', '--', '/tmp/tmp7talcY/mnt.0']
2020-03-05 21:03:29,726.726 INFO:__main__:kill 
2020-03-05 21:03:29,727.727 INFO:__main__:kill: killing pid 592 (['./bin/ceph-fuse', '-f', '--name', 'client.0', '/tmp/tmp7talcY/mnt.0'])
2020-03-05T21:03:22.940-0500 7fe4d97d4f80 -1 WARNING: all dangerous and experimental features are enabled.
2020-03-05T21:03:22.960-0500 7fe4d97d4f80 -1 WARNING: all dangerous and experimental features are enabled.
2020-03-05T21:03:22.964-0500 7fe4d97d4f80 -1 WARNING: all dangerous and experimental features are enabled.
2020-03-05T21:03:22.966-0500 7fe4d97d4f80 -1 init, newargv = 0x55a713dcb720 newargc=7
ceph-fuse[592]: starting ceph client
ceph-fuse[592]: starting fuse
ceph-fuse[592]: fuse finished with error 0 and tester_r 0
2020-03-05 21:03:29,730.730 INFO:__main__:Running ['rm', '-rf', '/tmp/tmp7talcY/mnt.0']
2020-03-05 21:03:29,735.735 INFO:__main__:Running ['stat', '--file-system', '--printf=%T\n', '--', '/tmp/tmp7talcY/mnt.1']
2020-03-05 21:03:29,743.743 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/tmp7talcY/mnt.1
2020-03-05 21:03:29,743.743 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on local...
2020-03-05 21:03:29,743.743 INFO:__main__:Running ['sudo', 'fusermount', '-u', '/tmp/tmp7talcY/mnt.1']
2020-03-05 21:03:29,807.807 INFO:__main__:Running ['stat', '--file-system', '--printf=%T\n', '--', '/tmp/tmp7talcY/mnt.1']
2020-03-05 21:03:29,817.817 INFO:__main__:kill 
2020-03-05 21:03:29,818.818 INFO:__main__:kill: killing pid 690 (['./bin/ceph-fuse', '-f', '--name', 'client.1', '/tmp/tmp7talcY/mnt.1'])
2020-03-05T21:03:25.246-0500 7fb2ab150f80 -1 WARNING: all dangerous and experimental features are enabled.
2020-03-05T21:03:25.265-0500 7fb2ab150f80 -1 WARNING: all dangerous and experimental features are enabled.
2020-03-05T21:03:25.271-0500 7fb2ab150f80 -1 WARNING: all dangerous and experimental features are enabled.
2020-03-05T21:03:25.273-0500 7fb2ab150f80 -1 init, newargv = 0x5555e8e7a830 newargc=7
ceph-fuse[690]: starting ceph client
ceph-fuse[690]: starting fuse
ceph-fuse[690]: fuse finished with error 0 and tester_r 0
2020-03-05 21:03:29,821.821 INFO:__main__:Running ['rm', '-rf', '/tmp/tmp7talcY/mnt.1']
2020-03-05 21:03:29,827.827 INFO:__main__:Running ['./bin/ceph', 'log', 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout']
2020-03-05 21:03:31,316.316 INFO:__main__:Stopped test: test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) in 27.633084s
2020-03-05 21:03:31,317.317 INFO:__main__:
2020-03-05 21:03:31,318.318 INFO:__main__:======================================================================
2020-03-05 21:03:31,318.318 INFO:__main__:ERROR: test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery)
2020-03-05 21:03:31,318.318 INFO:__main__:----------------------------------------------------------------------
2020-03-05 21:03:31,318.318 INFO:__main__:Traceback (most recent call last):
2020-03-05 21:03:31,319.319 INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/test_client_recovery.py", line 620, in test_config_session_timeout
2020-03-05 21:03:31,319.319 INFO:__main__:    self.fs.mds_asok(['session', 'config', '%s' % mount_a_gid, 'timeout', '%s' % (session_timeout * 2)])
2020-03-05 21:03:31,319.319 INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/filesystem.py", line 1001, in mds_asok
2020-03-05 21:03:31,319.319 INFO:__main__:    return self.json_asok(command, 'mds', mds_id, timeout=timeout)
2020-03-05 21:03:31,319.319 INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/filesystem.py", line 227, in json_asok
2020-03-05 21:03:31,320.320 INFO:__main__:    return json.loads(response_data)
2020-03-05 21:03:31,320.320 INFO:__main__:  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
2020-03-05 21:03:31,320.320 INFO:__main__:    return _default_decoder.decode(s)
2020-03-05 21:03:31,320.320 INFO:__main__:  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
2020-03-05 21:03:31,320.320 INFO:__main__:    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2020-03-05 21:03:31,321.321 INFO:__main__:  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
2020-03-05 21:03:31,321.321 INFO:__main__:    raise ValueError("No JSON object could be decoded")
2020-03-05 21:03:31,321.321 INFO:__main__:ValueError: No JSON object could be decoded
2020-03-05 21:03:31,321.321 INFO:__main__:
2020-03-05 21:03:31,322.322 INFO:__main__:----------------------------------------------------------------------
2020-03-05 21:03:31,322.322 INFO:__main__:Ran 1 test in 27.635s
2020-03-05 21:03:31,322.322 INFO:__main__:
2020-03-05 21:03:31,322.322 INFO:__main__:FAILED (errors=1)
2020-03-05 21:03:31,322.322 INFO:__main__:
2020-03-05 21:03:31,322.322 INFO:__main__:======================================================================
2020-03-05 21:03:31,322.322 INFO:__main__:ERROR: test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery)
2020-03-05 21:03:31,323.323 INFO:__main__:----------------------------------------------------------------------
2020-03-05 21:03:31,323.323 INFO:__main__:Traceback (most recent call last):
2020-03-05 21:03:31,323.323 INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/test_client_recovery.py", line 620, in test_config_session_timeout
2020-03-05 21:03:31,323.323 INFO:__main__:    self.fs.mds_asok(['session', 'config', '%s' % mount_a_gid, 'timeout', '%s' % (session_timeout * 2)])
2020-03-05 21:03:31,323.323 INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/filesystem.py", line 1001, in mds_asok
2020-03-05 21:03:31,323.323 INFO:__main__:    return self.json_asok(command, 'mds', mds_id, timeout=timeout)
2020-03-05 21:03:31,323.323 INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/filesystem.py", line 227, in json_asok
2020-03-05 21:03:31,324.324 INFO:__main__:    return json.loads(response_data)
2020-03-05 21:03:31,324.324 INFO:__main__:  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
2020-03-05 21:03:31,324.324 INFO:__main__:    return _default_decoder.decode(s)
2020-03-05 21:03:31,324.324 INFO:__main__:  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
2020-03-05 21:03:31,324.324 INFO:__main__:    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2020-03-05 21:03:31,324.324 INFO:__main__:  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
2020-03-05 21:03:31,324.324 INFO:__main__:    raise ValueError("No JSON object could be decoded")
2020-03-05 21:03:31,324.324 INFO:__main__:ValueError: No JSON object could be decoded
2020-03-05 21:03:31,324.324 INFO:__main__:

In vstart_runner.py it will filter out helper tools that don't exist in a vstart environment

args = [a for a in args if a not in {
             'adjust-ulimits', 'ceph-coverage', 'timeout'}]

So for the following command its 'timeout' option will be filter out too:

# ./bin/ceph daemon mds.b session config 8718 timeout 120

We need one way to fix this case.

#4 Updated by Xiubo Li 21 days ago

  • Status changed from In Progress to Fix Under Review

#5 Updated by Greg Farnum 4 days ago

  • Status changed from Fix Under Review to Resolved

#6 Updated by Greg Farnum 4 days ago

  • Status changed from Resolved to Pending Backport
  • Backport set to octopus

#7 Updated by Nathan Cutler about 21 hours ago

  • Copied to Backport #44844: octopus: qa:test_config_session_timeout failed with incorrect options added

Also available in: Atom PDF