Tools » teuthology

Trying to collect more data for this issue.
Here is the latest occurrence.

Last job ran -

/var/lib/teuthworker/archive/samuelj-2016-03-21_10:24:22-rados-wip-sam-testing-distro-basic-smithi/78108

2016-03-21 14:13:13,221.221 INFO:teuthology.nuke:targets:
  smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvBeXn0Zz5g95RMNbQRAo1U7huHmWgPl6u7Iiqx7cL+5Z1u4H4tULtB24YXYinAAesUo+4vHTqCK6qV4cp75XsIQRmUs+YR2PYrxlUNGSF9BgPWNEYT6EctW4xMjYskkW6RnWae7+GcupYxRgjDGzT8Ya8nltggnUtHU5cKcGr/9zN4IicHj9m8qR3O9KDimjnLmUrTXy9ZvHTSHJ6qGGmh4nM73fyitP0WJO4Fq2Khvkt901wrx3XQ8Yx9mF0kdsL+rPXA4IsO9K+2iCe8cKpsLDbWTOgd8NWNALmQqW6XFpdqyvW3e6pCiFvwMUMn9w4DMNhLtWYnMnUtrMwCTx7
2016-03-21 14:13:13,255.255 INFO:teuthology.nuke:checking console status of smithi067.ipmi.sepia.ceph.com
2016-03-21 14:13:13,537.537 INFO:teuthology.nuke:console ready on smithi067.ipmi.sepia.ceph.com
2016-03-21 14:13:13,538.538 INFO:teuthology.task.internal:Checking locks...
2016-03-21 14:13:13,575.575 INFO:teuthology.task.internal:Opening connections...
2016-03-21 14:13:13,857.857 INFO:teuthology.nuke:Clearing teuthology firewall rules...
2016-03-21 14:13:13,858.858 INFO:teuthology.orchestra.run.smithi067:Running: "sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore'" 
2016-03-21 14:13:13,925.925 INFO:teuthology.nuke:Cleared teuthology firewall rules.
2016-03-21 14:13:13,925.925 INFO:teuthology.nuke:Unmount ceph-fuse and killing daemons...
2016-03-21 14:13:13,925.925 INFO:teuthology.orchestra.run.smithi067:Running: "if grep -q ceph-fuse /etc/mtab ; then grep ceph-fuse /etc/mtab | grep -o ' /.* fuse' | grep -o '/.* ' | xargs -n 1 sudo fusermount -u ; fi ; if grep -q rbd-fuse /etc/mtab ; then grep rbd-fuse /etc/mtab | grep -o ' /.* fuse' | grep -o '/.* ' | xargs -n 1 sudo fusermount -u ; fi ; sudo killall --quiet ceph-mon ceph-osd ceph-mds ceph-fuse ceph-disk radosgw ceph_test_rados rados rbd-fuse apache2 || true" 
2016-03-21 14:13:13,938.938 INFO:teuthology.nuke:Waiting for ubuntu@smithi067.front.sepia.ceph.com to finish shutdowns...
2016-03-21 14:13:13,954.954 INFO:teuthology.nuke:All daemons killed.
2016-03-21 14:13:13,954.954 INFO:teuthology.nuke:Looking for kernel mounts to handle...
2016-03-21 14:13:13,954.954 INFO:teuthology.orchestra.run.smithi067:Running: "grep -q ' ceph ' /etc/mtab || grep -q '^/dev/rbd' /etc/mtab" 
2016-03-21 14:13:14,027.027 INFO:teuthology.nuke:Unmount any osd data directories...
2016-03-21 14:13:14,028.028 INFO:teuthology.orchestra.run.smithi067:Running: "grep /var/lib/ceph/osd/ /etc/mtab | awk '{print $2}' | xargs -r sudo umount ; true" 
2016-03-21 14:13:14,101.101 INFO:teuthology.nuke:Unmount any osd tmpfs dirs...
2016-03-21 14:13:14,101.101 INFO:teuthology.orchestra.run.smithi067:Running: "egrep 'tmpfs\\s+/mnt' /etc/mtab | awk '{print $2}' | xargs -r sudo umount ; true" 
2016-03-21 14:13:14,174.174 INFO:teuthology.nuke:Terminating Hadoop services...
2016-03-21 14:13:14,174.174 INFO:teuthology.orchestra.run.smithi067:Running: "ps -eo pid,cmd | grep 'java.*hadoop' | grep -v grep" 
2016-03-21 14:13:14,255.255 INFO:teuthology.nuke:All kernel mounts gone.
2016-03-21 14:13:14,255.255 INFO:teuthology.nuke:Synchronizing clocks...
2016-03-21 14:13:14,255.255 INFO:teuthology.nuke:Making sure firmware.git is not locked...
2016-03-21 14:13:14,255.255 INFO:teuthology.orchestra.run.smithi067:Running: 'sudo rm -f /lib/firmware/updates/.git/index.lock'
2016-03-21 14:13:14,334.334 INFO:teuthology.orchestra.run.smithi067:Running: 'rm -f /home/ubuntu/.cephdeploy.conf'
2016-03-21 14:13:14,401.401 INFO:teuthology.nuke:removing temporary configuration files on ubuntu@smithi067.front.sepia.ceph.com
2016-03-21 14:13:14,404.404 INFO:teuthology.nuke:Removing any multipath config/pkgs...
2016-03-21 14:13:14,405.405 INFO:teuthology.orchestra.run.smithi067:Running: 'sudo multipath -F'
2016-03-21 14:13:14,480.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: multipath: command not found
2016-03-21 14:13:14,480.480 INFO:teuthology.orchestra.run.smithi067:Running: 'sudo lsb_release -is'
2016-03-21 14:13:14,596.596 INFO:teuthology.task.install:Removing packages: multipath-tools on Debian system.
2016-03-21 14:13:14,597.597 INFO:teuthology.orchestra.run.smithi067:Running: 'for d in multipath-tools ; do sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" purge $d || true ; done'
2016-03-21 14:13:14,614.614 INFO:teuthology.orchestra.run.smithi067.stderr:E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
2016-03-21 14:13:14,616.616 INFO:teuthology.orchestra.run.smithi067:Running: "dpkg -l | grep '^.\\(U\\|H\\)R' | awk '{print $2}' | sudo xargs --no-run-if-empty dpkg -P --force-remove-reinstreq" 
2016-03-21 14:13:14,716.716 INFO:teuthology.orchestra.run.smithi067:Running: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-21 14:13:14,795.795 INFO:teuthology.orchestra.run.smithi067.stderr:E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
2016-03-21 14:13:14,797.797 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 404, in _remove_deb
    'autoremove',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on smithi067 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-21 14:13:14,800.800 ERROR:teuthology.nuke:Could not nuke {u'smithi067.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvBeXn0Zz5g95RMNbQRAo1U7huHmWgPl6u7Iiqx7cL+5Z1u4H4tULtB24YXYinAAesUo+4vHTqCK6qV4cp75XsIQRmUs+YR2PYrxlUNGSF9BgPWNEYT6EctW4xMjYskkW6RnWae7+GcupYxRgjDGzT8Ya8nltggnUtHU5cKcGr/9zN4IicHj9m8qR3O9KDimjnLmUrTXy9ZvHTSHJ6qGGmh4nM73fyitP0WJO4Fq2Khvkt901wrx3XQ8Yx9mF0kdsL+rPXA4IsO9K+2iCe8cKpsLDbWTOgd8NWNALmQqW6XFpdqyvW3e6pCiFvwMUMn9w4DMNhLtWYnMnUtrMwCTx7'}
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 648, in nuke_one
    nuke_helper(ctx, should_unlock)
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 740, in nuke_helper
    undo_multipath(ctx)
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 376, in undo_multipath
    "deb": ['multipath-tools'],
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 465, in remove_packages
    system_type], ctx, config, remote, pkgs[system_type])
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 404, in _remove_deb
    'autoremove',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on smithi067 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-21 14:13:14,805.805 ERROR:teuthology.nuke:Could not nuke the following targets:
targets:
  smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvBeXn0Zz5g95RMNbQRAo1U7huHmWgPl6u7Iiqx7cL+5Z1u4H4tULtB24YXYinAAesUo+4vHTqCK6qV4cp75XsIQRmUs+YR2PYrxlUNGSF9BgPWNEYT6EctW4xMjYskkW6RnWae7+GcupYxRgjDGzT8Ya8nltggnUtHU5cKcGr/9zN4IicHj9m8qR3O9KDimjnLmUrTXy9ZvHTSHJ6qGGmh4nM73fyitP0WJO4Fq2Khvkt901wrx3XQ8Yx9mF0kdsL+rPXA4IsO9K+2iCe8cKpsLDbWTOgd8NWNALmQqW6XFpdqyvW3e6pCiFvwMUMn9w4DMNhLtWYnMnUtrMwCTx7

This was the fix:

  yuriw@teuthology [14:13:16]> ssh smithi067
Warning: Permanently added 'smithi067,172.21.15.67' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-83-generic x86_64)

 * Documentation:  https://help.ubuntu.com/
Last login: Sat Mar 19 03:01:20 2016 from teuthology.front.sepia.ceph.com
yuriw@smithi067:~$ sudo apt-get install -f
E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem. 
yuriw@smithi067:~$ sudo dpkg --configure -a
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up libcephfs1 (10.0.5-2638-g7972c10-1trusty) ...
Setting up librados2 (10.0.5-2638-g7972c10-1trusty) ...
Processing triggers for ureadahead (0.100.0-16) ...
ureadahead will be reprofiled on next reboot
Setting up librbd1 (10.0.5-2638-g7972c10-1trusty) ...
Setting up python-cephfs (10.0.5-2638-g7972c10-1trusty) ...
Setting up python-rbd (10.0.5-2638-g7972c10-1trusty) ...
Setting up libradosstriper1 (10.0.5-2638-g7972c10-1trusty) ...
Setting up librgw2 (10.0.5-2638-g7972c10-1trusty) ...
Setting up python-rados (10.0.5-2638-g7972c10-1trusty) ...
Setting up ceph-common (10.0.5-2638-g7972c10-1trusty) ...
Setting system user ceph properties..usermod: no changes
..done
Processing triggers for ureadahead (0.100.0-16) ...
Setting up ceph-base (10.0.5-2638-g7972c10-1trusty) ...
ceph-all start/running
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
Processing triggers for ureadahead (0.100.0-16) ...

Last job ran:

smithi032

sage-2016-03-22_07:14:40-rados-wip-sage-testing-distro-basic-smithi/79866

2016-03-22 08:00:30,802.802 INFO:teuthology.orchestra.run.smithi032:Running: 'for d in multipath-tools ; do sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" purge $d || true ; done'
2016-03-22 08:00:30,818.818 INFO:teuthology.orchestra.run.smithi032.stderr:E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
2016-03-22 08:00:30,820.820 INFO:teuthology.orchestra.run.smithi032:Running: "dpkg -l | grep '^.\\(U\\|H\\)R' | awk '{print $2}' | sudo xargs --no-run-if-empty dpkg -P --force-remove-reinstreq" 
2016-03-22 08:00:30,918.918 INFO:teuthology.orchestra.run.smithi032:Running: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-22 08:00:30,996.996 INFO:teuthology.orchestra.run.smithi032.stderr:E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
2016-03-22 08:00:30,997.997 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 404, in _remove_deb
    'autoremove',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on smithi032 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-22 08:00:30,999.999 ERROR:teuthology.nuke:Could not nuke {u'smithi032.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2SKs5+hxiSd4RrrtKEjDR+EjJHJMH8dcMrss14arYK8CJQme4HCMVWjwJ0Mmncl7vVPjszx4GOohuzP075RFUAxYeLPf8jX4MSFqnGfHJONp2Jw6z2Sg0wbPdeJiUd+UeejF74GWMuN+7jb6G14EETBriBOMKDUHM5WPVjztGJ4NdUgrEPAHsdi3+SuV/qhSXCOSIA3/OlIBSRlU1L53zdWnse9Yc4mRC4ZeLuB4kxIH7oCQJHHVf74LJCDRs/AmFAp4hO9MVbicSCj4yAOm9Zd/ty94dCYNJlZqH8NoXcDewezG7xEz6nepdTc+lb0qq5rMFHLqZn/Avm50T7Lj9'}
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 648, in nuke_one
    nuke_helper(ctx, should_unlock)
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 740, in nuke_helper
    undo_multipath(ctx)
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 376, in undo_multipath
    "deb": ['multipath-tools'],
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 465, in remove_packages
    system_type], ctx, config, remote, pkgs[system_type])
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 404, in _remove_deb
    'autoremove',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on smithi032 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-22 08:00:31,002.002 ERROR:teuthology.nuke:Could not nuke the following targets:
targets:
  smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2SKs5+hxiSd4RrrtKEjDR+EjJHJMH8dcMrss14arYK8CJQme4HCMVWjwJ0Mmncl7vVPjszx4GOohuzP075RFUAxYeLPf8jX4MSFqnGfHJONp2Jw6z2Sg0wbPdeJiUd+UeejF74GWMuN+7jb6G14EETBriBOMKDUHM5WPVjztGJ4NdUgrEPAHsdi3+SuV/qhSXCOSIA3/OlIBSRlU1L53zdWnse9Yc4mRC4ZeLuB4kxIH7oCQJHHVf74LJCDRs/AmFAp4hO9MVbicSCj4yAOm9Zd/ty94dCYNJlZqH8NoXcDewezG7xEz6nepdTc+lb0qq5rMFHLqZn/Avm50T7Lj9

smithi054

teuthology-2016-03-19_22:00:02-rados-jewel-distro-basic-smithi/74869

2016-03-22 08:00:36,438.438 INFO:teuthology.orchestra.run.smithi054.stderr:E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
2016-03-22 08:00:36,440.440 INFO:teuthology.orchestra.run.smithi054:Running: "dpkg -l | grep '^.\\(U\\|H\\)R' | awk '{print $2}' | sudo xargs --no-run-if-empty dpkg -P --force-remove-reinstreq" 
2016-03-22 08:00:36,465.465 INFO:teuthology.nuke:Removing any multipath config/pkgs...
2016-03-22 08:00:36,465.465 INFO:teuthology.orchestra.run.smithi013:Running: 'sudo multipath -F'
2016-03-22 08:00:36,530.530 INFO:teuthology.orchestra.run.smithi013.stderr:sudo: multipath: command not found
2016-03-22 08:00:36,530.530 INFO:teuthology.orchestra.run.smithi013:Running: 'sudo lsb_release -is'
2016-03-22 08:00:36,539.539 INFO:teuthology.orchestra.run.smithi054:Running: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-22 08:00:36,619.619 INFO:teuthology.orchestra.run.smithi054.stderr:E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
2016-03-22 08:00:36,621.621 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 404, in _remove_deb
    'autoremove',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on smithi054 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'
2016-03-22 08:00:36,622.622 ERROR:teuthology.nuke:Could not nuke {u'smithi054.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHMIsYETP5DM1IDXroSAQH5ejn+KgEzjtsa2K2v6VLcKngmg/YwYs+tlu8Qeipy6n2HUHRXAI2c3IMFK2odfecps2oj2YwpPUtnn/KWgH9AHy+L9ku17y1R6LGye8anobaU1AkmbzwBzz4YLFCRHabdDh4jhrsXx4IaL2/OvfOXN3jHlmR7YEO4Xz8GiEvnAwKjYDV6v7DN2rXtcWPI3y2gac+A42n2a6CD3MET9FrktV3mN3HXydn8HnZ590U1XQNzW8/ebPRfx2kbG6OHu5exNjRxZRRYfKKtrG9qxvEHGrB7KtjTN2fOchUE0RfSLKu8St5/qAHBA5jD2EvcNDN'}
Traceback (most recent call last):
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 648, in nuke_one
    nuke_helper(ctx, should_unlock)
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 740, in nuke_helper
    undo_multipath(ctx)
  File "/home/yuriw/teuthology/teuthology/nuke.py", line 376, in undo_multipath
    "deb": ['multipath-tools'],
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 465, in remove_packages
    system_type], ctx, config, remote, pkgs[system_type])
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/yuriw/teuthology/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/yuriw/teuthology/teuthology/task/install.py", line 404, in _remove_deb
    'autoremove',
  File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/yuriw/teuthology/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on smithi054 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'

So the critical pieces of information about both of those are the run name, the host, and the error from apt, which is:

E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.

and that's two that are the same. Good to know. Now to figure out how to isolate why...