Project

General

Profile

Bug #21482

"file changed as we read it" in ceph-deploy-jewel

Added by Yuri Weinstein over 2 years ago. Updated over 1 year ago.

Status:
Can't reproduce
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-deploy
Pull request ID:
Crash signature:

Description

This is jewel v10.2.10

Run: http://pulpito.ceph.com/yuriw-2017-09-19_20:59:16-ceph-deploy-jewel-distro-basic-vps/
Jobs: 1650033 1650035
Logs: http://qa-proxy.ceph.com/teuthology/yuriw-2017-09-19_20:59:16-ceph-deploy-jewel-distro-basic-vps/1650033/teuthology.log

2017-09-19T21:20:19.152 INFO:teuthology.orchestra.run.vpm195:Running: "python -c 'import os; import tempfile; import sys;(fd,fname) = tempfile.mkstemp();os.close(fd);sys.stdout.write(fname.rstrip());sys.stdout.flush()'" 
2017-09-19T21:20:19.226 INFO:teuthology.orchestra.run.vpm195.stdout:/tmp/tmp_EIdi8
2017-09-19T21:20:19.226 INFO:teuthology.orchestra.run.vpm195:Running: 'sudo tar cz -f /tmp/tmp_EIdi8 -C /var/lib/ceph/mon -- .'
2017-09-19T21:20:19.335 INFO:teuthology.orchestra.run.vpm195.stderr:tar: ./ceph-vpm195/store.db/000006.log: file changed as we read it
2017-09-19T21:20:19.338 INFO:tasks.ceph_deploy:Removing ceph-deploy ...
2017-09-19T21:20:19.338 INFO:teuthology.orchestra.run.vpm181:Running: 'rm -rf /home/ubuntu/cephtest/ceph-deploy'
2017-09-19T21:20:19.416 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
2017-09-19T21:20:19.417 INFO:teuthology.orchestra.run.vpm181:Running: 'sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits'
2017-09-19T21:20:19.424 INFO:teuthology.orchestra.run.vpm195:Running: 'sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits'
2017-09-19T21:20:19.485 ERROR:teuthology.run_tasks:Manager failed: ceph-deploy
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 159, in run_tasks
    suppress = manager.__exit__(*exc_info)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/teuthworker/src/github.com_ceph_ceph_jewel/qa/tasks/ceph_deploy.py", line 694, in task
    yield
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 46, in nested
    if exit(*exc):
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/teuthworker/src/github.com_ceph_ceph_jewel/qa/tasks/ceph_deploy.py", line 422, in build_ceph_cluster
    path + '/' + role + '.tgz')
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/misc.py", line 790, in pull_directory_tarball
    remote.get_tar(remotedir, localfile, sudo=True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 367, in get_tar
    self.run(args=args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 423, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 155, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 177, in _raise_for_status
    node=self.hostname, label=self.label
CommandFailedError: Command failed on vpm195 with status 1: 'sudo tar cz -f /tmp/tmp_EIdi8 -C /var/lib/ceph/mon -- .'
2017-09-19T21:20:19.486 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys
2017-09-19T21:20:19.521 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
    yield vars
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/ssh_keys.py", line 206, in task
    yield
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 159, in run_tasks
    suppress = manager.__exit__(*exc_info)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/teuthworker/src/github.com_ceph_ceph_jewel/qa/tasks/ceph_deploy.py", line 694, in task
    yield
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 46, in nested
    if exit(*exc):
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/teuthworker/src/github.com_ceph_ceph_jewel/qa/tasks/ceph_deploy.py", line 422, in build_ceph_cluster
    path + '/' + role + '.tgz')
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/misc.py", line 790, in pull_directory_tarball
    remote.get_tar(remotedir, localfile, sudo=True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 367, in get_tar
    self.run(args=args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 423, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 155, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 177, in _raise_for_status
    node=self.hostname, label=self.label
CommandFailedError: Command failed on vpm195 with status 1: 'sudo tar cz -f /tmp/tmp_EIdi8 -C /var/lib/ceph/mon -- .'

Related issues

Related to Ceph - Bug #15098: "stop ceph-all" no longer stops ceph Resolved 03/11/2016

History

#1 Updated by Yuri Weinstein over 2 years ago

  • Related to Bug #15098: "stop ceph-all" no longer stops ceph added

#3 Updated by Sage Weil over 1 year ago

  • Status changed from New to Can't reproduce

Also available in: Atom PDF