Project

General

Profile

Bug #21326

xenial/4.4.0-93-generic rbd: sysfs write failed map failed: (110) Connection timed out

Added by Vasu Kulkarni over 6 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

Test: description: smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore.yaml
tasks/rbd_workunit_suites_iozone.yaml}

Default features : 5
Logs: http://qa-proxy.ceph.com/teuthology/vasu-2017-09-08_19:37:31-smoke-luminous-distro-basic-ovh/1610427/teuthology.log

If it was a feature issue, i should have seen a feature mismatch warning.

2017-09-08T20:36:55.460 INFO:tasks.rbd:Creating image testimage.client.0 with size 20480
2017-09-08T20:36:55.462 INFO:teuthology.orchestra.run.ovh044:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rbd -p rbd create --size 20480 testimage.client.0'
2017-09-08T20:36:56.989 INFO:tasks.rbd:Loading rbd kernel module...
2017-09-08T20:36:57.265 INFO:teuthology.orchestra.run.ovh044:Running: 'sudo modprobe rbd'
2017-09-08T20:36:57.382 INFO:tasks.rbd:Creating rbd block devices...
2017-09-08T20:36:57.534 INFO:teuthology.orchestra.run.ovh044:Running: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rbd --user 0 -p rbd map testimage.client.0 && while test '!' -e /dev/rbd/rbd/testimage.client.0 ; do sleep 1 ; done" 
2017-09-08T20:37:15.333 INFO:teuthology.orchestra.run.ovh018:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2017-09-08T20:37:15.404 INFO:teuthology.orchestra.run.ovh044:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2017-09-08T20:37:15.467 INFO:teuthology.orchestra.run.ovh100:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2017-09-08T20:37:45.546 INFO:teuthology.orchestra.run.ovh018:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2017-09-08T20:37:45.606 INFO:teuthology.orchestra.run.ovh044:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2017-09-08T20:37:45.667 INFO:teuthology.orchestra.run.ovh100:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2017-09-08T20:37:57.694 INFO:teuthology.orchestra.run.ovh044.stderr:rbd: sysfs write failed
2017-09-08T20:37:57.784 INFO:teuthology.orchestra.run.ovh044.stdout:In some cases useful info is found in syslog - try "dmesg | tail".
2017-09-08T20:37:57.787 INFO:teuthology.orchestra.run.ovh044.stderr:rbd: map failed: (110) Connection timed out
2017-09-08T20:37:57.790 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 30, in nested
    vars.append(enter())
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/github.com_ceph_ceph_luminous/qa/tasks/rbd.py", line 257, in dev_create
    'done',
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 423, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 155, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 177, in _raise_for_status
    node=self.hostname, label=self.label
CommandFailedError: Command failed on ovh044 with status 110: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rbd --user 0 -p rbd map testimage.client.0 && while test '!' -e /dev/rbd/rbd/testimage.client.0 ; do sleep 1 ; done" 

History

#1 Updated by Ilya Dryomov over 6 years ago

It was a cluster-level feature mismatch, not image-level. 4.4.0-93-generic doesn't support CRUSH_TUNABLES5:

2017-09-08T20:36:57.684990+00:00 ovh044 kernel: [  668.672021] libceph: mon0 158.69.91.212:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400000000000000
2017-09-08T20:36:57.685016+00:00 ovh044 kernel: [  668.674581] libceph: mon0 158.69.91.212:6789 missing required protocol features

#2 Updated by Ilya Dryomov over 6 years ago

  • Status changed from In Progress to Closed

Also available in: Atom PDF