Project

General

Profile

Bug #46878

Command failed on XXX with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic'

Added by Brad Hubbard over 3 years ago. Updated over 1 year ago.

Status:
Can't reproduce
Priority:
High
Assignee:
Category:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

/a/yuriw-2020-08-06_00:31:28-rados-wip-yuri8-testing-octopus-distro-basic-smithi/5291079

2020-08-06T15:32:09.935 INFO:teuthology.run_tasks:Running task kernel...
2020-08-06T15:32:09.955 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}}
2020-08-06T15:32:09.955 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}}, timeout 300
2020-08-06T15:32:09.956 INFO:teuthology.orchestra.run.smithi140:> uname -r
2020-08-06T15:32:09.962 INFO:teuthology.orchestra.run.smithi140.stdout:4.15.0-112-generic
2020-08-06T15:32:09.962 INFO:teuthology.task.kernel:Running kernel on smithi140: 4.15.0-112-generic
2020-08-06T15:32:09.963 INFO:teuthology.orchestra.run.smithi140:> sudo apt-get clean
2020-08-06T15:32:10.076 INFO:teuthology.orchestra.run.smithi140:> sudo apt-get update
2020-08-06T15:32:10.200 INFO:teuthology.orchestra.run.smithi140.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
2020-08-06T15:32:10.326 INFO:teuthology.orchestra.run.smithi140.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
2020-08-06T15:32:10.407 INFO:teuthology.orchestra.run.smithi140.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
2020-08-06T15:32:10.681 INFO:teuthology.orchestra.run.smithi140.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
2020-08-06T15:32:11.043 INFO:teuthology.orchestra.run.smithi140.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [808 kB]
2020-08-06T15:32:11.145 INFO:teuthology.orchestra.run.smithi140.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [516 kB]
2020-08-06T15:32:11.154 INFO:teuthology.orchestra.run.smithi140.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [254 kB]
2020-08-06T15:32:11.159 INFO:teuthology.orchestra.run.smithi140.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [689 kB]
2020-08-06T15:32:11.173 INFO:teuthology.orchestra.run.smithi140.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [632 kB]
2020-08-06T15:32:11.184 INFO:teuthology.orchestra.run.smithi140.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [228 kB]
2020-08-06T15:32:11.330 INFO:teuthology.orchestra.run.smithi140.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,032 kB]
2020-08-06T15:32:11.695 INFO:teuthology.orchestra.run.smithi140.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [723 kB]
2020-08-06T15:32:11.720 INFO:teuthology.orchestra.run.smithi140.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,097 kB]
2020-08-06T15:32:11.799 INFO:teuthology.orchestra.run.smithi140.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,027 kB]
2020-08-06T15:32:12.311 INFO:teuthology.orchestra.run.smithi140.stdout:Fetched 7,259 kB in 2s (3,673 kB/s)
2020-08-06T15:32:13.552 INFO:teuthology.orchestra.run.smithi140.stdout:Reading package lists...
2020-08-06T15:32:13.600 INFO:teuthology.orchestra.run.smithi140:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic
2020-08-06T15:32:13.647 INFO:teuthology.orchestra.run.smithi140.stderr:E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-08-06T15:32:13.647 INFO:teuthology.orchestra.run.smithi140.stderr:E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-08-06T15:32:13.648 DEBUG:teuthology.orchestra.run:got remote process result: 100
2020-08-06T15:32:13.649 INFO:teuthology.orchestra.run.smithi140:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic
2020-08-06T15:32:13.707 INFO:teuthology.orchestra.run.smithi140.stderr:E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-08-06T15:32:13.707 INFO:teuthology.orchestra.run.smithi140.stderr:E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-08-06T15:32:13.708 DEBUG:teuthology.orchestra.run:got remote process result: 100
2020-08-06T15:32:13.709 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 1094, in get_latest_image_version_deb
    'linux-image-current-generic'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 204, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 446, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi140 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic'

Perhaps we need to check if we can acquire the lock somehow in 'get_latest_image_version_deb' before calling 'sudo apt-get update'?

History

#1 Updated by Deepika Upadhyay over 3 years ago

/a/yuriw-2020-08-20_19:48:15-rados-wip-yuri-testing-2020-08-17-1723-octopus-distro-basic-smithi/5363036

#2 Updated by Deepika Upadhyay over 3 years ago

/a/yuriw-2020-08-26_18:16:40-rados-wip-yuri-testing-2020-08-26-1631-octopus-distro-basic-smithi/5378314
/a/yuriw-2020-08-27_00:49:53-rados-wip-yuri8-testing-2020-08-26-2329-octopus-distro-basic-smithi/5379163

#3 Updated by Patrick Donnelly over 3 years ago

  • Priority changed from Normal to High

/ceph/teuthology-archive/pdonnell-2020-10-22_01:17:59-fs-wip-pdonnell-testing-20201021.232150-distro-basic-smithi/5546171/teuthology.log

#4 Updated by Patrick Donnelly over 3 years ago

Brad Hubbard wrote:

/a/yuriw-2020-08-06_00:31:28-rados-wip-yuri8-testing-octopus-distro-basic-smithi/5291079

[...]

Perhaps we need to check if we can acquire the lock somehow in 'get_latest_image_version_deb' before calling 'sudo apt-get update'?

I think just a retry for `apt-get` if it gets EAGAIN would be suitable.

#5 Updated by Xiubo Li over 3 years ago

  • Status changed from New to In Progress
  • Assignee set to Xiubo Li

#6 Updated by Xiubo Li over 3 years ago

  • Status changed from In Progress to Fix Under Review

#7 Updated by Xiubo Li over 3 years ago

As Kefu suggested we will keep the tracker opened and when it's reproduced again we need to find the root cause. Thanks.

#8 Updated by Deepika Upadhyay over 2 years ago

2021-07-27T06:52:20.175 INFO:teuthology.orchestra.run.smithi013.stderr:E: Unable to locate package linux-image-current-generic
2021-07-27T06:52:20.177 DEBUG:teuthology.orchestra.run:got remote process result: 100

http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-26_16:38:53-rbd-wip-yuri3-testing-2021-07-26-0800-pacific-distro-basic-smithi/6294949/teuthology.log

@Xiubo ^

#9 Updated by Josh Durgin over 2 years ago

  • Status changed from Fix Under Review to Can't reproduce

#10 Updated by Matan Breizman over 1 year ago

/a/yuriw-2022-11-28_16:28:53-rados-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/7094549

Also available in: Atom PDF