Actions
Bug #10803
closedmachines would not nuke, require apt-get -f manual run (e.g. librados2 librados2-dbg librbd1 python-ceph)
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Running teuthology-nuke --stale --owner scheduled_teuthology@teuthology --unlock
would not complete on plana59:
2015-02-09 08:28:19,854.854 INFO:teuthology.orchestra.run.plana59.stderr:Errors were encountered while processing: 2015-02-09 08:28:19,854.854 INFO:teuthology.orchestra.run.plana59.stderr: librbd1 2015-02-09 08:28:19,855.855 ERROR:teuthology.parallel:Exception in parallel execution Traceback (most recent call last): File "/home/ubuntu/teuthology/teuthology/parallel.py", line 82, in __exit__ for result in self: File "/home/ubuntu/teuthology/teuthology/parallel.py", line 101, in next resurrect_traceback(result) File "/home/ubuntu/teuthology/teuthology/parallel.py", line 19, in capture_traceback return func(*args, **kwargs) File "/home/ubuntu/teuthology/teuthology/task/install.py", line 581, in _remove_deb 'dpkg', '-P', '--force-remove-reinstreq', File "/home/ubuntu/teuthology/teuthology/orchestra/remote.py", line 128, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/ubuntu/teuthology/teuthology/orchestra/run.py", line 378, in run r.wait() File "/home/ubuntu/teuthology/teuthology/orchestra/run.py", line 114, in wait label=self.label) CommandFailedError: Command failed on plana59 with status 123: "dpkg -l | grep '^.HR' | awk '{print $2}' | sudo xargs --no-run-if-empty dpkg -P --force-remove-reinstreq" 2015-02-09 08:28:19,878.878 ERROR:teuthology.nuke:Could not nuke all targets in {u'plana59.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaHG83zRXo6ydv6IGWDFTf6YNjWG9M5LRbbYIpPXKOqCg9zfI/4ZjymLpznESFIACVrqe06jqD7uvsQPOlbcm3W/H44su70C21KrzMs77IpskMT7tYgCzY75uxbwg949qYIRf1SEY2RW0Bf2zldbOeKAY/TcnGIkLtc4NCIDPfCxMG0rAJJgUAwbvbKVUqLKe/jcyu3RiiAxV3TGjTAzTz+XHwT46gDXB5Fxt49Sfx+AgpILHk7DvN/HILtU3gRT9ac0D2WlQi1sJLDgjeTAZxyfpRR5iZH4tWYBFIS7C4ugHYye95zUYTc/3Jt364Jl/giUherGjE5od7p65VjxRJ'} Traceback (most recent call last): File "/home/ubuntu/teuthology/teuthology/nuke.py", line 446, in nuke_one nuke_helper(ctx, should_unlock) File "/home/ubuntu/teuthology/teuthology/nuke.py", line 536, in nuke_helper remove_installed_packages(ctx) File "/home/ubuntu/teuthology/teuthology/nuke.py", line 264, in remove_installed_packages ['salt-common', 'salt-minion', 'calamari-server']}) File "/home/ubuntu/teuthology/teuthology/task/install.py", line 650, in remove_packages system_type], ctx, config, remote, pkgs[system_type]) File "/home/ubuntu/teuthology/teuthology/parallel.py", line 82, in __exit__ for result in self: File "/home/ubuntu/teuthology/teuthology/parallel.py", line 101, in next resurrect_traceback(result) File "/home/ubuntu/teuthology/teuthology/parallel.py", line 19, in capture_traceback return func(*args, **kwargs) File "/home/ubuntu/teuthology/teuthology/task/install.py", line 581, in _remove_deb 'dpkg', '-P', '--force-remove-reinstreq', File "/home/ubuntu/teuthology/teuthology/orchestra/remote.py", line 128, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/ubuntu/teuthology/teuthology/orchestra/run.py", line 378, in run r.wait() File "/home/ubuntu/teuthology/teuthology/orchestra/run.py", line 114, in wait label=self.label) CommandFailedError: Command failed on plana59 with status 123: "dpkg -l | grep '^.HR' | awk '{print $2}' | sudo xargs --no-run-if-empty dpkg -P --force-remove-reinstreq"
and running thso most like will fix the issue:
ubuntu@plana59:~$ sudo apt-get -f install Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following extra packages will be installed: librados2 librados2-dbg librbd1 python-ceph The following packages will be upgraded: librados2 librados2-dbg librbd1 python-ceph 4 upgraded, 0 newly installed, 0 to remove and 189 not upgraded. 6 not fully installed or removed. Need to get 0 B/30.3 MB of archives. After this operation, 15.7 MB of additional disk space will be used. Do you want to continue [Y/n]?
Updated by Zack Cerza about 9 years ago
How does apt-get -f install
fix the issue? Doesn't that still leave ceph packages behind?
Updated by Zack Cerza about 9 years ago
Also I think we need more output to actually diagnose the problem. The traceback isn't enough when a command fails; we need the actual error from the command output.
Updated by Zack Cerza about 9 years ago
As I said, the traceback is not enough. We need to know why the command failed, so we need its output.
Updated by Yuri Weinstein about 9 years ago
Zack, here it is:
2015-02-09 08:28:17,737.737 INFO:teuthology.orchestra.run.plana59.stdout:Reading package lists... 2015-02-09 08:28:18,056.056 INFO:teuthology.orchestra.run.plana59.stdout:Building dependency tree... 2015-02-09 08:28:18,058.058 INFO:teuthology.orchestra.run.plana59.stdout:Reading state information... 2015-02-09 08:28:18,151.151 INFO:teuthology.orchestra.run.plana59.stdout:You might want to run 'apt-get -f install' to correct these: 2015-02-09 08:28:18,151.151 INFO:teuthology.orchestra.run.plana59.stdout:The following packages have unmet dependencies: 2015-02-09 08:28:18,152.152 INFO:teuthology.orchestra.run.plana59.stdout: ceph-common : Depends: python-ceph (= 0.92-920-ga08d3be-1precise) but 0.80.8-22-g26966c4-1precise is to be installed 2015-02-09 08:28:18,152.152 INFO:teuthology.orchestra.run.plana59.stdout: librbd1 : Depends: librados2 (= 0.92-920-ga08d3be-1precise) but 0.80.8-22-g26966c4-1precise is to be installed 2015-02-09 08:28:18,172.172 INFO:teuthology.orchestra.run.plana59.stderr:E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution). 2015-02-09 08:28:18,186.186 INFO:teuthology.orchestra.run.plana59.stdout:Reading package lists... 2015-02-09 08:28:18,489.489 INFO:teuthology.orchestra.run.plana59.stdout:Building dependency tree... 2015-02-09 08:28:18,491.491 INFO:teuthology.orchestra.run.plana59.stdout:Reading state information... 2015-02-09 08:28:18,577.577 INFO:teuthology.orchestra.run.plana59.stderr:E: Unable to locate package salt-common 2015-02-09 08:28:18,589.589 INFO:teuthology.orchestra.run.plana59.stdout:Reading package lists... 2015-02-09 08:28:18,903.903 INFO:teuthology.orchestra.run.plana59.stdout:Building dependency tree... 2015-02-09 08:28:18,905.905 INFO:teuthology.orchestra.run.plana59.stdout:Reading state information... 2015-02-09 08:28:19,025.025 INFO:teuthology.orchestra.run.plana59.stderr:E: Unable to locate package salt-minion 2015-02-09 08:28:19,042.042 INFO:teuthology.orchestra.run.plana59.stdout:Reading package lists... 2015-02-09 08:28:19,437.437 INFO:teuthology.orchestra.run.plana59.stdout:Building dependency tree... 2015-02-09 08:28:19,439.439 INFO:teuthology.orchestra.run.plana59.stdout:Reading state information... 2015-02-09 08:28:19,559.559 INFO:teuthology.orchestra.run.plana59.stderr:E: Unable to locate package calamari-server 2015-02-09 08:28:19,561.561 INFO:teuthology.orchestra.run.plana59:Running: "dpkg -l | grep '^.HR' | awk '{print $2}' | sudo xargs --no-run-if-empty dpkg -P --force-remove-reinstreq" 2015-02-09 08:28:19,753.753 INFO:teuthology.orchestra.run.plana59.stderr:dpkg: dependency problems prevent removal of librbd1: 2015-02-09 08:28:19,753.753 INFO:teuthology.orchestra.run.plana59.stderr: ceph-test depends on librbd1; however: 2015-02-09 08:28:19,754.754 INFO:teuthology.orchestra.run.plana59.stderr: Package librbd1 is to be removed. 2015-02-09 08:28:19,754.754 INFO:teuthology.orchestra.run.plana59.stderr: qemu-kvm depends on librbd1; however: 2015-02-09 08:28:19,754.754 INFO:teuthology.orchestra.run.plana59.stderr: Package librbd1 is to be removed. 2015-02-09 08:28:19,754.754 INFO:teuthology.orchestra.run.plana59.stderr: qemu-utils depends on librbd1. 2015-02-09 08:28:19,754.754 INFO:teuthology.orchestra.run.plana59.stderr: python-ceph depends on librbd1. 2015-02-09 08:28:19,755.755 INFO:teuthology.orchestra.run.plana59.stderr: tgt depends on librbd1. 2015-02-09 08:28:19,755.755 INFO:teuthology.orchestra.run.plana59.stderr:dpkg: error processing librbd1 (--purge): 2015-02-09 08:28:19,755.755 INFO:teuthology.orchestra.run.plana59.stderr: dependency problems - not removing
Updated by Andrew Schoen about 9 years ago
- Assignee changed from Sandon Van Ness to Andrew Schoen
Updated by Andrew Schoen about 9 years ago
- Status changed from New to Resolved
Actions