Ceph » RADOS

Run: http://pulpito.ceph.com/teuthology-2019-01-25_02:30:02-upgrade:mimic-x-master-distro-basic-smithi/
Jobs: all
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2019-01-25_02:30:02-upgrade:mimic-x-master-distro-basic-smithi/3503936/teuthology.log

2019-01-27T22:56:45.148 INFO:teuthology.orchestra.run.smithi005:Running:
2019-01-27T22:56:45.148 INFO:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph health
2019-01-27T22:56:45.355 INFO:teuthology.misc.health.smithi005.stderr:2019-01-27 22:56:45.370 7fe2bc22f700 -1 WARNING: all dangerous and experimental features are enabled.
2019-01-27T22:56:45.399 INFO:teuthology.misc.health.smithi005.stderr:2019-01-27 22:56:45.414 7fe2b904e700 -1 --2- v1:172.21.15.5:34842/2723700930 >> v2:172.21.15.164:6790/0 conn(0x7fe2ac0e0e90 0x7fe2ac0df1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_addrvec_and_identify decode peer addr failed
2019-01-27T22:56:45.426 INFO:teuthology.misc.health.smithi005.stderr:2019-01-27 22:56:45.441 7fe2bc22f700 -1 WARNING: all dangerous and experimental features are enabled.
2019-01-27T22:56:45.427 INFO:teuthology.misc.health.smithi005.stderr:2019-01-27 22:56:45.441 7fe2bc22f700 -1 asok(0x7fe2b4000bf0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.48378.asok': (13) Permission denied
2019-01-27T22:56:45.760 INFO:teuthology.misc.health.smithi005.stdout:HEALTH_WARN 3 monitors have not enabled msgr2
2019-01-27T22:56:45.770 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN 3 monitors have not enabled msgr2
2019-01-27T22:56:46.771 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 86, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 65, in run_one_task
    return task(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/parallel.py", line 56, in task
    p.spawn(_run_spawned, ctx, confg, taskname)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 85, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 99, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 22, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/parallel.py", line 64, in _run_spawned
    mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 65, in run_one_task
    return task(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/sequential.py", line 49, in task
    mgr.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/git.ceph.com_ceph_master/qa/tasks/ceph.py", line 1644, in restart
    healthy(ctx=ctx, config=dict(cluster=cluster))
  File "/home/teuthworker/src/git.ceph.com_ceph_master/qa/tasks/ceph.py", line 1489, in healthy
    ceph_cluster=cluster_name,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/misc.py", line 936, in wait_until_healthy
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 132, in __call__
    raise MaxWhileTries(error_msg)
MaxWhileTries: 'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds