Project

General

Profile

Actions

Bug #8396

closed

osd: message delayed in Session misdirected after split

Added by Tamilarasi muthamizhan almost 10 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

logs:ubuntu@teuthology:/a/teuthology-2014-05-18_02:30:21-rados-master-testing-basic-plana/260734

ceph version 0.80-469-g991f7f1

2014-05-18T11:36:19.886 INFO:teuthology.orchestra.run.plana39.stderr:config set <var> <val> [<val>...]
2014-05-18T11:36:19.886 INFO:teuthology.orchestra.run.plana39.stderr:version
2014-05-18T11:36:19.887 INFO:teuthology.orchestra.run.plana39.stderr:2
2014-05-18T11:36:19.887 INFO:teuthology.orchestra.run.plana39.stderr:config get <var>
2014-05-18T11:36:19.887 INFO:teuthology.orchestra.run.plana39.stderr:0
2014-05-18T11:36:19.888 INFO:teuthology.orchestra.run.plana39.stderr:admin_socket: invalid command
2014-05-18T11:36:19.891 INFO:teuthology.task.thrashosds.ceph_manager:waiting on admin_socket for 3, ['dump_ops_in_flight']
2014-05-18T11:36:20.140 INFO:teuthology.task.ceph.osd.3.plana39.stderr:2014-05-18 11:36:20.139436 7f38ca252780 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2014-05-18T11:36:24.745 INFO:teuthology.task.rados.rados.0.plana59.stderr:3885: Error: oid 432 read returned error code -2
2014-05-18T11:36:24.794 INFO:teuthology.task.rados.rados.0.plana59.stderr:./test/osd/RadosModel.h: In function 'virtual void ReadOp::_finish(TestOp::CallbackInfo*)' thread 7fcf5dffb700 time 2014-05-18 11:36:24.743525
2014-05-18T11:36:24.794 INFO:teuthology.task.rados.rados.0.plana59.stderr:./test/osd/RadosModel.h: 1045: FAILED assert(0)
2014-05-18T11:36:24.833 INFO:teuthology.task.rados.rados.0.plana59.stderr: ceph version 0.80-469-g991f7f1 (991f7f15a6e107b33a24bbef1169f21eb7fcce2c)
2014-05-18T11:36:24.833 INFO:teuthology.task.rados.rados.0.plana59.stderr: 1: (ReadOp::_finish(TestOp::CallbackInfo*)+0x12f) [0x41b03f]
2014-05-18T11:36:24.833 INFO:teuthology.task.rados.rados.0.plana59.stderr: 2: (librados::C_AioComplete::finish(int)+0x1d) [0x7fcf68cb694d]
2014-05-18T11:36:24.833 INFO:teuthology.task.rados.rados.0.plana59.stderr: 3: (Context::complete(int)+0x9) [0x7fcf68c94409]
2014-05-18T11:36:24.834 INFO:teuthology.task.rados.rados.0.plana59.stderr: 4: (Finisher::finisher_thread_entry()+0x1c0) [0x7fcf68d47490]
2014-05-18T11:36:24.834 INFO:teuthology.task.rados.rados.0.plana59.stderr: 5: (()+0x7e9a) [0x7fcf688ebe9a]
2014-05-18T11:36:24.834 INFO:teuthology.task.rados.rados.0.plana59.stderr: 6: (clone()+0x6d) [0x7fcf681023fd]
2014-05-18T11:36:24.834 INFO:teuthology.task.rados.rados.0.plana59.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

archive_path: /var/lib/teuthworker/archive/teuthology-2014-05-18_02:30:21-rados-master-testing-basic-plana/260734
branch: master
description: rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml
  thrashers/pggrow.yaml workloads/readwrite.yaml}
email: null
job_id: '260734'
kernel:
  kdb: true
  sha1: 335cb91ce950ce0e12294af671c64a468d89194c
last_in_suite: false
machine_type: plana
name: teuthology-2014-05-18_02:30:21-rados-master-testing-basic-plana
nuke-on-error: true
os_type: ubuntu
overrides:
  admin_socket:
    branch: master
  ceph:
    conf:
      global:
        ms inject delay max: 1
        ms inject delay probability: 0.005
        ms inject delay type: osd
        ms inject internal delays: 0.002
        ms inject socket failures: 2500
      mon:
        debug mon: 20
        debug ms: 1
        debug paxos: 20
      osd:
        debug filestore: 20
        debug journal: 20
        debug ms: 1
        debug osd: 20
        osd sloppy crc: true
    fs: xfs
    log-whitelist:
    - slow request
    sha1: 991f7f15a6e107b33a24bbef1169f21eb7fcce2c
  ceph-deploy:
    branch:
      dev: master
    conf:
      client:
        log file: /var/log/ceph/ceph-$name.$pid.log
      mon:
        debug mon: 1
        debug ms: 20
        debug paxos: 20
        osd default pool size: 2
  install:
    ceph:
      sha1: 991f7f15a6e107b33a24bbef1169f21eb7fcce2c
  s3tests:
    branch: master
  workunit:
    sha1: 991f7f15a6e107b33a24bbef1169f21eb7fcce2c
owner: scheduled_teuthology@teuthology
roles:
- - mon.a
  - mon.c
  - osd.0
  - osd.1
  - osd.2
  - client.0
- - mon.b
  - mds.a
  - osd.3
  - osd.4
  - osd.5
  - client.1
suite: rados
tasks:
- chef: null
- clock.check: null
- install: null
- ceph:
    log-whitelist:
    - wrongly marked me down
    - objects unfound and apparently lost
- thrashosds:
    chance_pgnum_grow: 2
    chance_pgpnum_fix: 1
    timeout: 1200
- rados:
    clients:
    - client.0
    objects: 500
    op_weights:
      delete: 10
      read: 45
      write: 45
    ops: 4000
teuthology_branch: master
verbose: true


Related issues 3 (0 open3 closed)

Has duplicate Ceph - Bug #8522: ENOENT on read from ceph_test_rados + osd thrashing Duplicate06/03/2014

Actions
Has duplicate Ceph - Bug #8613: ENOENT from ceph_test_radosDuplicate06/16/2014

Actions
Blocks Ceph - Bug #8505: OSD osd/OSD.cc: 6222: FAILED assert(p->second.empty())ResolvedSamuel Just06/02/2014

Actions
Actions

Also available in: Atom PDF