Project

General

Profile

Actions

Bug #15430

closed

"HEALTH_WARN 1 pgs stuck inactive.." in upgrade:firefly-hammer-x-infernalis-distro-basic-openstack

Added by Yuri Weinstein about 8 years ago. Updated about 7 years ago.

Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
upgrade/firefly-hammer-x
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ovh.sepia.ceph.com:8081/teuthology-2016-04-07_20:30:08-upgrade:firefly-hammer-x-infernalis-distro-basic-openstack/
Job: 30609
Logs: http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-04-07_20:30:08-upgrade:firefly-hammer-x-infernalis-distro-basic-openstack/30609/teuthology.log

osd pool get test-rados-api-target094088.ovh.sepia.ceph.com-18183-19 pg_num'
2016-04-07T21:54:38.820 INFO:teuthology.orchestra.run.target094086.stderr:Error ENOENT: unrecognized pool 'test-rados-api-target094088.ovh.sepia.ceph.com-18183-19'
2016-04-07T21:54:38.832 INFO:tasks.cephfs.filesystem.ceph_manager:Failed to get pg_num from pool test-rados-api-target094088.ovh.sepia.ceph.com-18183-19, ignoring
2016-04-07T21:54:38.834 INFO:teuthology.orchestra.run.target094086:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph mds dump --format=json'
....
2016-04-07T22:18:50.317 INFO:teuthology.orchestra.run.target094086:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph health'
2016-04-07T22:18:50.757 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 1 requests are blocked > 32 sec
2016-04-07T22:18:57.758 INFO:teuthology.orchestra.run.target094086:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph health'
2016-04-07T22:18:58.199 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 1 requests are blocked > 32 sec
2016-04-07T22:19:05.202 INFO:teuthology.orchestra.run.target094086:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph health'
2016-04-07T22:19:05.660 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 1 requests are blocked > 32 sec
2016-04-07T22:19:12.664 INFO:teuthology.orchestra.run.target094086:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph health'
2016-04-07T22:19:13.095 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 1 requests are blocked > 32 sec
2016-04-07T22:19:20.097 INFO:teuthology.orchestra.run.target094086:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph health'
2016-04-07T22:19:20.544 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 1 requests are blocked > 32 sec
2016-04-08T00:53:32.256 INFO:tasks.workunit:Stopping ['rados/test-upgrade-v9.0.1.sh', 'cls'] on client.0...
2016-04-08T00:53:32.258 INFO:teuthology.orchestra.run.target094088:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/workunit.client.0 /home/ubuntu/cephtest/clone'
2016-04-08T00:53:32.525 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/ceph-qa-suite_infernalis/tasks/workunit.py", line 385, in _run_tests
    label="workunit test {workunit}".format(workunit=workunit)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed (workunit test rados/test-upgrade-v9.0.1.sh) on target094088 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=hammer TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/rados/test-upgrade-v9.0.1.sh'
Actions

Also available in: Atom PDF