Project

General

Profile

Bug #10716

"FATAL: Chef::Exceptions::ChildConvergeError" in Octo

Added by Yuri Weinstein about 9 years ago. Updated about 9 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Sandon Van Ness
Category:
-
Target version:
-
% Done:

100%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Many tests failed with this.
One for example: http://magna002.ceph.redhat.com/teuthology-2015-02-02_02:32:01-rados-giant-distro-basic-magna/37312/

2015-02-02T06:19:27.359 INFO:teuthology.orchestra.run.magna034.stdout:[2015-02-02T06:19:25-05:00] ERROR: Running exception handlers
2015-02-02T06:19:27.360 INFO:teuthology.orchestra.run.magna034.stdout:[2015-02-02T06:19:25-05:00] ERROR: Exception handlers complete
2015-02-02T06:19:27.400 INFO:teuthology.orchestra.run.magna034.stdout:[2015-02-02T06:19:25-05:00] FATAL: Stacktrace dumped to /tmp/solo-from-scratch.KWT75pyVpN6L/solo/chef-stacktrace.out
2015-02-02T06:19:27.401 INFO:teuthology.orchestra.run.magna034.stdout:[2015-02-02T06:19:25-05:00] ERROR: execute[Getting around redhat blocking of NTP] (ceph-qa::ntp-deb line 16) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 3600s:
2015-02-02T06:19:27.441 INFO:teuthology.orchestra.run.magna034.stdout:Command execeded allowed execution time, killed by TERM signal.
2015-02-02T06:19:27.442 INFO:teuthology.orchestra.run.magna034.stdout:---- Begin output of       sudo sed -i 's;^server [A-Z:a-z:.:-:0-9]*;server clock.corp.redhat.com;g' /etc/ntp.conf || true
2015-02-02T06:19:27.482 INFO:teuthology.orchestra.run.magna034.stdout:      sudo ntpd -gq
2015-02-02T06:19:27.483 INFO:teuthology.orchestra.run.magna034.stdout: ----
2015-02-02T06:19:27.523 INFO:teuthology.orchestra.run.magna034.stdout:STDOUT:
2015-02-02T06:19:27.524 INFO:teuthology.orchestra.run.magna034.stdout:STDERR:
2015-02-02T06:19:27.564 INFO:teuthology.orchestra.run.magna034.stdout:---- End output of       sudo sed -i 's;^server [A-Z:a-z:.:-:0-9]*;server clock.corp.redhat.com;g' /etc/ntp.conf || true
2015-02-02T06:19:27.565 INFO:teuthology.orchestra.run.magna034.stdout:      sudo ntpd -gq
2015-02-02T06:19:27.605 INFO:teuthology.orchestra.run.magna034.stdout: ----
2015-02-02T06:19:27.606 INFO:teuthology.orchestra.run.magna034.stdout:Ran       sudo sed -i 's;^server [A-Z:a-z:.:-:0-9]*;server clock.corp.redhat.com;g' /etc/ntp.conf || true
2015-02-02T06:19:27.646 INFO:teuthology.orchestra.run.magna034.stdout:      sudo ntpd -gq
2015-02-02T06:19:27.648 INFO:teuthology.orchestra.run.magna034.stdout: returned
2015-02-02T06:19:27.687 INFO:teuthology.orchestra.run.magna034.stdout:[2015-02-02T06:19:25-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
2015-02-02T06:19:27.689 INFO:teuthology.orchestra.run.magna034.stderr:+ cleanup
2015-02-02T06:19:27.728 INFO:teuthology.orchestra.run.magna034.stderr:+ rm -rf /tmp/solo-from-scratch.KWT75pyVpN6L
2015-02-02T06:19:27.729 INFO:teuthology.orchestra.run.magna041.stdout:[2015-02-02T06:19:24-05:00] ERROR: execute[Getting around redhat blocking of NTP] (ceph-qa::ntp-deb line 16) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 3600s:
2015-02-02T06:19:27.770 INFO:teuthology.orchestra.run.magna041.stdout:Command execeded allowed execution time, killed by TERM signal.
2015-02-02T06:19:27.771 INFO:teuthology.orchestra.run.magna041.stdout:---- Begin output of       sudo sed -i 's;^server [A-Z:a-z:.:-:0-9]*;server clock.corp.redhat.com;g' /etc/ntp.conf || true
2015-02-02T06:19:27.772 INFO:teuthology.orchestra.run.magna041.stdout:      sudo ntpd -gq
2015-02-02T06:19:27.812 INFO:teuthology.orchestra.run.magna041.stdout: ----
2015-02-02T06:19:27.814 INFO:teuthology.orchestra.run.magna041.stdout:STDOUT:
2015-02-02T06:19:27.854 INFO:teuthology.orchestra.run.magna041.stdout:STDERR:
2015-02-02T06:19:27.855 INFO:teuthology.orchestra.run.magna041.stdout:---- End output of       sudo sed -i 's;^server [A-Z:a-z:.:-:0-9]*;server clock.corp.redhat.com;g' /etc/ntp.conf || true
2015-02-02T06:19:27.895 INFO:teuthology.orchestra.run.magna041.stdout:      sudo ntpd -gq
2015-02-02T06:19:27.897 INFO:teuthology.orchestra.run.magna041.stdout: ----
2015-02-02T06:19:27.936 INFO:teuthology.orchestra.run.magna041.stdout:Ran       sudo sed -i 's;^server [A-Z:a-z:.:-:0-9]*;server clock.corp.redhat.com;g' /etc/ntp.conf || true
2015-02-02T06:19:27.937 INFO:teuthology.orchestra.run.magna041.stdout:      sudo ntpd -gq
2015-02-02T06:19:27.939 INFO:teuthology.orchestra.run.magna041.stdout: returned
2015-02-02T06:19:27.979 INFO:teuthology.orchestra.run.magna041.stdout:[2015-02-02T06:19:24-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
2015-02-02T06:19:27.981 INFO:teuthology.orchestra.run.magna041.stderr:+ cleanup
2015-02-02T06:19:28.021 INFO:teuthology.orchestra.run.magna041.stderr:+ rm -rf /tmp/solo-from-scratch.k2FlmJjEe8Sn
2015-02-02T06:19:28.022 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 53, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 41, in run_one_task
    return fn(**kwargs)
  File "/home/teuthworker/src/ceph-qa-suite_giant/tasks/chef.py", line 29, in task
    wait=False,
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 391, in wait
    proc.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 106, in wait
    exitstatus=status, node=self.hostname)
CommandFailedError: Command failed on magna034 with status 1: "wget -O- 'http://ceph.com/git/?p=ceph-qa-chef.git;a=blob_plain;f=solo/solo-from-scratch;hb=HEAD' | sh -x" 

History

#1 Updated by Sandon Van Ness about 9 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

This was fixed on 02/03 by ceph-qa-chef commit:

0414cc827909f7f6e8fda25055b5c3cc58bd7861

Also available in: Atom PDF