Infrastructure » sepia

2016-12-02 17:23:03,301.301 INFO:teuthology.orchestra.run.mira097.stderr:rm: cannot remove ‘/var/lib/ceph’: Device or resource busy
2016-12-02 17:23:03,328.328 INFO:teuthology.orchestra.run.mira097.stderr:rm: cannot remove ‘/var/lib/ceph’: Device or resource busy
2016-12-02 17:23:03,329.329 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install/__init__.py", line 93, in _purge_data
    'rm', '-rf', '--one-file-system', '--', '/var/lib/ceph',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 192, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 403, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 166, in wait
    label=self.label)
CommandFailedError: Command failed on mira097 with status 1: "sudo rm -rf --one-file-system -- /var/lib/ceph || true ; test -d /var/lib/ceph && sudo find /var/lib/ceph -mindepth 1 -maxdepth 2 -type d -exec umount '{}' ';' ; sudo rm -rf --one-file-system -- /var/lib/ceph" 
2016-12-02 17:23:03,330.330 ERROR:teuthology.nuke:Could not nuke {u'mira097.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDG2dOJzrUSyviG/hX39DV044Im1qvLw+u9bdgGZCaTpJrl2up9lPP7tVKhRFQ2I9i0CRZG+7nZiWH4WicZt6+5tFaPyJpZZJQgyaS9DVd9mOdSVtFzTcMCStztr8ULWjf7MA+c385u5UbW6gxYfpAZIVTpGhIiPAUDOvFP8CNtDusGoB685pKs4vOtppLBEoiRjC78kJ3mAV1omOkTr52+38reZ22FchlaQpkmZXNJ0YM4KVNnMwXuYl5KJpnj3Z4dYcyJqDTSdVw373uoklZwQ11ogRMmQISQzaBjfIbOx8bqSrlm5iMSwvYtDC3V3VjTG0jE5japdK1HNl7g2WkH'}
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/nuke/__init__.py", line 281, in nuke_one
    nuke_helper(ctx, should_unlock)
  File "/home/yuriw/teuthology/teuthology/nuke/__init__.py", line 339, in nuke_helper
    remove_ceph_data(ctx)
  File "/home/yuriw/teuthology/teuthology/nuke/actions.py", line 332, in remove_ceph_data
    install_task.purge_data(ctx)
  File "/home/yuriw/teuthology/teuthology/task/install/__init__.py", line 67, in purge_data
    p.spawn(_purge_data, remote)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install/__init__.py", line 93, in _purge_data
    'rm', '-rf', '--one-file-system', '--', '/var/lib/ceph',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 192, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 403, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 166, in wait
    label=self.label)
CommandFailedError: Command failed on mira097 with status 1: "sudo rm -rf --one-file-system -- /var/lib/ceph || true ; test -d /var/lib/ceph && sudo find /var/lib/ceph -mindepth 1 -maxdepth 2 -type d -exec umount '{}' ';' ; sudo rm -rf --one-file-system -- /var/lib/ceph" 
2016-12-02 17:23:03,331.331 ERROR:teuthology.nuke:Could not nuke the following targets:
targets:
  mira097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDG2dOJzrUSyviG/hX39DV044Im1qvLw+u9bdgGZCaTpJrl2up9lPP7tVKhRFQ2I9i0CRZG+7nZiWH4WicZt6+5tFaPyJpZZJQgyaS9DVd9mOdSVtFzTcMCStztr8ULWjf7MA+c385u5UbW6gxYfpAZIVTpGhIiPAUDOvFP8CNtDusGoB685pKs4vOtppLBEoiRjC78kJ3mAV1omOkTr52+38reZ22FchlaQpkmZXNJ0YM4KVNnMwXuYl5KJpnj3Z4dYcyJqDTSdVw373uoklZwQ11ogRMmQISQzaBjfIbOx8bqSrlm5iMSwvYtDC3V3VjTG0jE5japdK1HNl7g2WkH