Project

General

Profile

Actions

Bug #5754

closed

mon: occasional osd full map not trimmed

Added by Sage Weil over 10 years ago. Updated over 10 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Monitor
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

seems to happen after sync doing catchup, but the txn dumps so far confirm it is removed.

reproduced about 20% of the time on this job:

machine_type: plana
nuke-on-error: true
overrides:
  admin_socket:
    branch: next
  ceph:
    conf:
      global:
        ms inject socket failures: 5000
      mon:
        debug mon: 50
        debug ms: 1
        debug paxos: 50
        mon min osdmap epochs: 25
        paxos service trim min: 5
    fs: xfs
    log-whitelist:
    - slow request
  install:
    ceph:
      branch: wip-paxos
  s3tests:
    branch: next
  workunit:
    branch: next
roles:
- - mon.a
  - mon.b
  - mon.c
  - mon.d
  - mon.e
  - osd.0
  - osd.1
  - osd.2
- - mon.f
  - mon.g
  - mon.h
  - mon.i
  - mds.a
  - osd.3
  - osd.4
  - osd.5
  - client.0
tasks:
- chef: null
- clock.check: null
- install: null
- ceph: null
- mon_thrash:
    revive_delay: 90
    thrash_delay: 1
    thrash_many: true
    thrash_store: true
- rados:
    clients:
    - client.0
    objects: 50
    op_weights:
      delete: 50
      read: 100
      rollback: 50
      snap_create: 50
      snap_remove: 50
      write: 100
    ops: 4000
teuthology_branch: next


Related issues 1 (0 open1 closed)

Related to Ceph - Bug #5832: mon scrub mismatchDuplicate08/01/2013

Actions
Actions #1

Updated by Ian Colle over 10 years ago

  • Subject changed from mon: occasinoal osd full map not trimmed to mon: occasional osd full map not trimmed
Actions #2

Updated by Sage Weil over 10 years ago

a mon transaction dump clearly shows the key being removed, and it is then visible later.

other evidence that this is a leveldb bug: disabling 'mon compact on trim' makes it go away.

next step is probably to install a raring backport on precise and see if it is fixed...

Actions #4

Updated by Sage Weil over 10 years ago

  • Status changed from In Progress to 15
Actions #5

Updated by Sage Weil over 10 years ago

  • Priority changed from Urgent to High
Actions #6

Updated by Sage Weil over 10 years ago

  • Status changed from 15 to Resolved
Actions

Also available in: Atom PDF