Project

General

Profile

Actions

Bug #3041

closed

ceph manager down during osd recovery

Added by Tamilarasi muthamizhan over 11 years ago. Updated about 11 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Logs: ubuntu@teuthology:/a/teuthology-2012-08-23_19:00:08-regression-master-testing-gcov/7533

2012-08-24T08:43:56.194 INFO:teuthology.task.osd_recovery.ceph_manager:active or down!
2012-08-24T08:43:56.194 DEBUG:teuthology.orchestra.run:Running: 'LD_LIBRARY_PRELOAD=/tmp/cephtest/binary/usr/local/lib /tmp/cephtest/enable-coredump /tmp/cephtest/binary/usr/local/bin/ceph-coverage /tmp/cephtest/archive/coverage /tmp/cephtest/binary/usr/local/bin/ceph -k /tmp/cephtest/ceph.keyring -c /tmp/cephtest/ceph.conf --concise -- pg dump --format=json'
2012-08-24T08:43:56.319 ERROR:teuthology.run_tasks:Saw exception from tasks
Traceback (most recent call last):
  File "/var/lib/teuthworker/teuthology/teuthology/run_tasks.py", line 25, in run_tasks
    manager = _run_one_task(taskname, ctx=ctx, config=config)
  File "/var/lib/teuthworker/teuthology/teuthology/run_tasks.py", line 14, in _run_one_task
    return fn(**kwargs)
  File "/var/lib/teuthworker/teuthology/teuthology/task/osd_recovery.py", line 174, in test_incomplete_pgs
    assert manager.get_num_down() > 0
AssertionError
2012-08-24T08:43:56.433 DEBUG:teuthology.run_tasks:Unwinding manager <contextlib.GeneratorContextManager object at 0x15f9a10>
2012-08-24T08:43:56.433 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/var/lib/teuthworker/teuthology/teuthology/contextutil.py", line 27, in nested
    yield vars
  File "/var/lib/teuthworker/teuthology/teuthology/task/ceph.py", line 1077, in task
    yield
  File "/var/lib/teuthworker/teuthology/teuthology/run_tasks.py", line 25, in run_tasks
    manager = _run_one_task(taskname, ctx=ctx, config=config)
  File "/var/lib/teuthworker/teuthology/teuthology/run_tasks.py", line 14, in _run_one_task
    return fn(**kwargs)
  File "/var/lib/teuthworker/teuthology/teuthology/task/osd_recovery.py", line 174, in test_incomplete_pgs
    assert manager.get_num_down() > 0
AssertionError

ubuntu@teuthology:/a/teuthology-2012-08-23_19:00:08-regression-master-testing-gcov/7533$ cat config.yaml 
kernel: &id001
  kdb: true
  sha1: 532b65ecb2fd9e6cab5f8586a1191a4408d9afbc
nuke-on-error: true
overrides:
  ceph:
    conf:
      global:
        ms inject socket failures: 5000
    coverage: true
    fs: btrfs
    log-whitelist:
    - slow request
    sha1: a5901c6d6cb6b77f0bb8ee5cb91d06a51355811f
  workunit:
    sha1: a5901c6d6cb6b77f0bb8ee5cb91d06a51355811f
roles:
- - mon.a
  - mon.b
  - mon.c
  - mds.a
  - osd.0
  - osd.1
  - osd.2
  - osd.3
targets:
  ubuntu@plana40.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEwyNlwC9Utqf3PCjL2JR4wwDkzpdEJuW93DOW82vYVisYEGod454JwXeNkjqzTUk6tXeRoUM9f/C6sZS3LFgHcMYt6m0sxP8DC4qU+q0YxCw9zLY8bXKe4DDjijM62h/SnyqyOWIh9amGT7wRwZEHBV1BKvZbNxQIJ7ESkuKsk/tJfWKhq7dSw6E/+MZ4yQtXvTyaJ3pK96Hq2uoUkawv+FxXBrzG3FtTTYA8gqA1SIiV3erEIQuBK/WD74i5yK4rwpfGTo7jNc0V6wrwO1BKFj/OGjSC+2LSAkBgf8WLe6UL/dHr3bBEyzm0V4xMf5Iqb8JGvkaXNEfbFqzKC2Wv
tasks:
- internal.lock_machines: 1
- internal.save_config: null
- internal.check_lock: null
- internal.connect: null
- internal.check_conflict: null
- kernel: *id001
- internal.base: null
- internal.archive: null
- internal.coredump: null
- internal.syslog: null
- internal.timer: null
- chef: null
- clock: null
- ceph:
    conf:
      osd:
        osd min pg log entries: 5
    log-whitelist:
    - wrongly marked me down
- osd_recovery.test_incomplete_pgs: null
ubuntu@teuthology:/a/teuthology-2012-08-23_19:00:08-regression-master-testing-gcov/7533$ cat summary.yaml 
ceph-sha1: a5901c6d6cb6b77f0bb8ee5cb91d06a51355811f
description: collection:rados-singleton all:osd-recovery-incomplete.yaml fs:btrfs.yaml
  msgr-failures:few.yaml
duration: 177.62271308898926
failure_reason: ''
flavor: gcov
mon.a-kernel-sha1: 532b65ecb2fd9e6cab5f8586a1191a4408d9afbc
owner: scheduled_teuthology@teuthology
success: false
Actions #1

Updated by Tamilarasi muthamizhan over 11 years ago

ubuntu@teuthology:/a/teuthology-2012-10-06_00:00:05-regression-next-testing-basic/2831

Actions #2

Updated by Sage Weil about 11 years ago

  • Status changed from New to Resolved
Actions

Also available in: Atom PDF