Bug #44201
Updated by Laura Paduano about 4 years ago
2020-02-18T21:39:49.633 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/3834979578 shutdown_connections
2020-02-18T21:39:49.633 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/3834979578 wait complete.
2020-02-18T21:39:49.633 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:6800/24427,v1:172.21.15.121:6801/24427] conn(0x559db6b9b6e0 msgr2=0x559db6b9dad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2020-02-18T21:39:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:6800/24427,v1:172.21.15.121:6801/24427] conn(0x559db6b9b6e0 0x559db6b9dad0 crc :-1 s=READY pgs=449 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> [v2:172.21.15.191:6808/25230,v1:172.21.15.191:6809/25230] conn(0x559db6ba1e90 msgr2=0x559db6ba42c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2020-02-18T21:39:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.191:6808/25230,v1:172.21.15.191:6809/25230] conn(0x559db6ba1e90 0x559db6ba42c0 crc :-1 s=READY pgs=374 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> [v2:172.21.15.191:6800/25229,v1:172.21.15.191:6801/25229] conn(0x559db6ba5ec0 msgr2=0x559db6ba82b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2020-02-18T21:39:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.191:6800/25229,v1:172.21.15.191:6801/25229] conn(0x559db6ba5ec0 0x559db6ba82b0 crc :-1 s=READY pgs=378 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> [v2:172.21.15.191:6816/25245,v1:172.21.15.191:6817/25245] conn(0x559db6b9e930 msgr2=0x559db6ba0d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.191:6816/25245,v1:172.21.15.191:6817/25245] conn(0x559db6b9e930 0x559db6ba0d20 crc :-1 s=READY pgs=445 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff344fb0700 1 -- 172.21.15.121:0/2645806808 reap_dead start
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 10 mgrc shutdown
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff344fb0700 1 -- 172.21.15.121:0/2645806808 reap_dead start
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:6802/109841,v1:172.21.15.121:6803/109841] conn(0x7ff2880539b0 msgr2=0x7ff288055e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:6802/109841,v1:172.21.15.121:6803/109841] conn(0x7ff2880539b0 0x7ff288055e70 crc :-1 s=READY pgs=26 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.644 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559db6b6d440 msgr2=0x559db6b4bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2020-02-18T21:39:49.645 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559db6b6d440 0x559db6b4bb80 secure :-1 s=READY pgs=4772 cs=0 l=1 rx=0x7ff338037000 tx=0x7ff338037cc0).stop
2020-02-18T21:39:49.645 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 shutdown_connections
2020-02-18T21:39:49.645 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559db6b6d440 0x559db6b4bb80 unknown :-1 s=CLOSED pgs=4772 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.645 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 --2- 172.21.15.121:0/2645806808 >> [v2:172.21.15.121:6802/109841,v1:172.21.15.121:6803/109841] conn(0x7ff2880539b0 0x7ff288055e70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rx=0 tx=0).stop
2020-02-18T21:39:49.645 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 >> 172.21.15.121:0/2645806808 conn(0x559db6b73c00 msgr2=0x559db6b50750 unknown :-1 s=STATE_NONE l=0).mark_down
2020-02-18T21:39:49.645 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 shutdown_connections
2020-02-18T21:39:49.646 INFO:teuthology.orchestra.run.smithi121.stderr:2020-02-18T21:39:49.556+0000 7ff356fbe180 1 -- 172.21.15.121:0/2645806808 wait complete.
2020-02-18T21:39:49.653 DEBUG:teuthology.orchestra.run:got remote process result: 17
2020-02-18T21:39:49.677 INFO:tasks.cephfs_test_runner:ERROR
2020-02-18T21:39:49.677 INFO:tasks.cephfs_test_runner:
2020-02-18T21:39:49.677 INFO:tasks.cephfs_test_runner:======================================================================
2020-02-18T21:39:49.677 INFO:tasks.cephfs_test_runner:ERROR: setUpClass (tasks.mgr.dashboard.test_rgw.RgwBucketTest)
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/test_rgw.py", line 116, in setUpClass
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner: super(RgwBucketTest, cls).setUpClass()
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/test_rgw.py", line 25, in setUpClass
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner: '--system', '--access-key', 'admin', '--secret', 'admin'
2020-02-18T21:39:49.678 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/helper.py", line 413, in _radosgw_admin_cmd
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: cls._cmd(args)
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/helper.py", line 401, in _cmd
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: return cls.mgr_cluster.admin_remote.run(args=args)
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: r.wait()
2020-02-18T21:39:49.679 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait
2020-02-18T21:39:49.680 INFO:tasks.cephfs_test_runner: self._raise_for_status()
2020-02-18T21:39:49.680 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status
2020-02-18T21:39:49.680 INFO:tasks.cephfs_test_runner: node=self.hostname, label=self.label
2020-02-18T21:39:49.680 INFO:tasks.cephfs_test_runner:CommandFailedError: Command failed on smithi121 with status 17: 'radosgw-admin user create --uid admin --display-name admin --system --access-key admin --secret admin'
2020-02-18T21:39:49.680 INFO:tasks.cephfs_test_runner:
2020-02-18T21:39:49.680 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-02-18T21:39:49.687 INFO:tasks.cephfs_test_runner:Ran 169 tests in 1832.099s
2020-02-18T21:39:49.687 INFO:tasks.cephfs_test_runner:
2020-02-18T21:39:49.687 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner:
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner:======================================================================
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner:ERROR: setUpClass (tasks.mgr.dashboard.test_rgw.RgwBucketTest)
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/test_rgw.py", line 116, in setUpClass
2020-02-18T21:39:49.688 INFO:tasks.cephfs_test_runner: super(RgwBucketTest, cls).setUpClass()
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/test_rgw.py", line 25, in setUpClass
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: '--system', '--access-key', 'admin', '--secret', 'admin'
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/helper.py", line 413, in _radosgw_admin_cmd
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: cls._cmd(args)
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/mgr/dashboard/helper.py", line 401, in _cmd
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: return cls.mgr_cluster.admin_remote.run(args=args)
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run
2020-02-18T21:39:49.689 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner: r.wait()
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner: self._raise_for_status()
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner: node=self.hostname, label=self.label
2020-02-18T21:39:49.694 INFO:tasks.cephfs_test_runner:CommandFailedError: Command failed on smithi121 with status 17: 'radosgw-admin user create --uid admin --display-name admin --system --access-key admin --secret admin'
2020-02-18T21:39:49.695 INFO:tasks.cephfs_test_runner:
2020-02-18T21:39:49.695 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks
manager.__enter__()
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/cephfs_test_runner.py", line 211, in task
raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: setUpClass (tasks.mgr.dashboard.test_rgw.RgwBucketTest)
2020-02-18T21:39:49.701 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner
2020-02-18T21:39:49.746 DEBUG:teuthology.run_tasks:Unwinding manager rgw
2020-02-18T21:39:49.804 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 34, in nested
yield vars
File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/rgw.py", line 392, in task
yield
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks
manager.__enter__()
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-32552-32829-33178/qa/tasks/cephfs_test_runner.py", line 211, in task
raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: setUpClass (tasks.mgr.dashboard.test_rgw.RgwBucketTest)
2020-02-18T21:39:49.818 DEBUG:tasks.rgw.client.0:waiting for process to exit
See:
* http://pulpito.ceph.com/laura-2020-02-18_20:49:28-rados:dashboard-wip-laura-testing-32552-32829-33178-distro-basic-smithi/
* https://jenkins.ceph.com/view/all/job/ceph-api-nightly-master-backend/78/consoleFull
* http://pulpito.ceph.com/sage-2020-02-19_01:44:50-rados:dashboard-wip-sage-testing-2020-02-18-1551-distro-basic-smithi/