Bug #11126: plana94 rsyslog won't start - sepia - Ceph

Actions

Copy link

Bug #11126

closed

plana94 rsyslog won't start

Added by Yuri Weinstein about 9 years ago. Updated over 8 years ago.

Status:

Duplicate

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Crash signature (v1):

Crash signature (v2):

Description

Lots of runs effected by this machine.

One for example http://qa-proxy.ceph.com/teuthology/teuthology-2015-03-15_17:10:01-upgrade:dumpling-firefly-x-hammer-distro-basic-multi/805080/teuthology.log

rsyslog restart'
2015-03-15T20:30:24.637 INFO:teuthology.orchestra.run.plana79.stdout:rsyslog stop/waiting
2015-03-15T20:30:24.638 INFO:teuthology.orchestra.run.plana94.stderr:stop: Unknown instance:
2015-03-15T20:30:24.639 INFO:teuthology.orchestra.run.plana07.stdout:rsyslog stop/waiting
2015-03-15T20:30:24.652 INFO:teuthology.orchestra.run.plana79.stdout:rsyslog start/running, process 2121
2015-03-15T20:30:24.654 INFO:teuthology.orchestra.run.plana94.stderr:start: Job failed to start
2015-03-15T20:30:24.657 INFO:teuthology.orchestra.run.plana07.stdout:rsyslog start/running, process 27608
2015-03-15T20:30:24.657 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 55, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/internal.py", line 564, in syslog
    wait=False,
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 401, in wait
    proc.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on plana94 with status 1: 'sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart'
2015-03-15T20:30:24.693 ERROR:teuthology.run_tasks: Sentry event: http://sentry.ceph.com/sepia/teuthology/search?q=ae97feae5e9e41698c49b3f116e81a54
CommandFailedError: Command failed on plana94 with status 1: 'sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart'

Some runs:
http://pulpito.ceph.com/teuthology-2015-03-15_17:10:01-upgrade:dumpling-firefly-x-hammer-distro-basic-multi/
http://pulpito.ceph.com/teuthology-2015-03-15_18:13:01-upgrade:firefly-x-giant-distro-basic-multi/
http://pulpito.ceph.com/teuthology-2015-03-15_17:05:01-upgrade:giant-x-hammer-distro-basic-multi/

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Yuri Weinstein about 9 years ago

Description updated (diff)

Actions

Copy link

Updated by Zack Cerza about 9 years ago

Subject changed from plana94 "Job failed to start" failing jobs to plana94 rsyslog won't start

2015-03-15T20:30:24.617 INFO:teuthology.orchestra.run.plana94:Running: 'sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart'
2015-03-15T20:30:24.638 INFO:teuthology.orchestra.run.plana94.stderr:stop: Unknown instance:
2015-03-15T20:30:24.654 INFO:teuthology.orchestra.run.plana94.stderr:start: Job failed to start

Actions

Copy link

Updated by Dan Mick almost 9 years ago

Wow: this is in /var/log/upstart/rsyslog.log:

Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.Feature buffer full.

Actions

Copy link