Project

General

Profile

Actions

Bug #5020

closed

osd: 2.5 deep-scrub stat mismatch, got 710/665 objects, 0/0 clones, 2908160/2723840 bytes

Added by Tamilarasi muthamizhan almost 11 years ago. Updated almost 11 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

ubuntu@teuthology:/a/teuthology-2013-05-10_01:00:06-rados-master-testing-basic/10305


ubuntu@teuthology:/a/teuthology-2013-05-10_01:00:06-rados-master-testing-basic/10305$ cat orig.config.yaml 
kernel:
  kdb: true
  sha1: b5b09be30cf99f9c699e825629f02e3bce555d44
machine_type: plana
nuke-on-error: true
overrides:
  ceph:
    conf:
      global:
        ms inject socket failures: 5000
      mon:
        debug mon: 20
        debug ms: 20
        debug paxos: 20
      osd:
        osd op thread timeout: 60
    fs: btrfs
    log-whitelist:
    - slow request
    sha1: b353da6f682d223ba14812da0fe814eca72ad6f5
  s3tests:
    branch: master
  workunit:
    sha1: b353da6f682d223ba14812da0fe814eca72ad6f5
roles:
- - mon.a
  - mon.b
  - mon.c
  - mds.a
  - osd.0
  - osd.1
  - osd.2
tasks:
- chef: null
- clock.check: null
- install: null
- ceph:
    conf:
      osd:
        osd min pg log entries: 5
    log-whitelist:
    - wrongly marked me down
- osd_recovery: null

2013-05-10 08:45:52.559532 7f6237e01700  0 osd.1 22 do_command r=0 
2013-05-10 08:45:55.825333 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 994a5f05/benchmark_data_plana20_7011_object4047/head//2
2013-05-10 08:45:55.892178 7f6238602700  0 log [ERR] : 2.5 osd.1 missing d14ee115/benchmark_data_plana20_7011_object4243/head//2
2013-05-10 08:45:55.892183 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 7b0f1415/benchmark_data_plana20_7011_object4125/head//2
2013-05-10 08:45:56.289271 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 90d6da35/benchmark_data_plana20_7011_object4201/head//2
2013-05-10 08:45:56.394378 7f6238602700  0 log [ERR] : 2.5 osd.1 missing a65ac245/benchmark_data_plana20_7011_object4072/head//2
2013-05-10 08:45:56.505997 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 74560d45/benchmark_data_plana20_7011_object4103/head//2
2013-05-10 08:45:56.531935 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 10dbaf45/benchmark_data_plana20_7011_object4045/head//2
2013-05-10 08:45:56.658261 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 9e65ed55/benchmark_data_plana20_7011_object4069/head//2
2013-05-10 08:45:56.658265 7f6238602700  0 log [ERR] : 2.5 osd.1 missing f5b6f065/benchmark_data_plana20_7011_object4329/head//2
2013-05-10 08:45:56.731999 7f6238602700  0 log [ERR] : 2.5 osd.1 missing efeae865/benchmark_data_plana20_7011_object4277/head//2
2013-05-10 08:45:56.732002 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 22b15c65/benchmark_data_plana20_7011_object3974/head//2
2013-05-10 08:45:56.808959 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 8b880875/benchmark_data_plana20_7011_object4094/head//2
2013-05-10 08:45:56.848746 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 28693b75/benchmark_data_plana20_7011_object4272/head//2
2013-05-10 08:45:56.931933 7f6238602700  0 log [ERR] : 2.5 osd.1 missing e2ce9f75/benchmark_data_plana20_7011_object4306/head//2
2013-05-10 08:45:57.048653 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 7822cf85/benchmark_data_plana20_7011_object3958/head//2
2013-05-10 08:45:57.136690 7f6238602700  0 log [ERR] : 2.5 osd.1 missing e9667ca5/benchmark_data_plana20_7011_object4334/head//2
2013-05-10 08:45:57.353029 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 6a0886d5/benchmark_data_plana20_7011_object4182/head//2
2013-05-10 08:45:57.452920 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 8e4256e5/benchmark_data_plana20_7011_object4006/head//2
2013-05-10 08:45:57.452923 7f6238602700  0 log [ERR] : 2.5 osd.1 missing d1fc87e5/benchmark_data_plana20_7011_object4255/head//2
2013-05-10 08:45:57.475535 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 5afb7ae5/benchmark_data_plana20_7011_object4108/head//2
2013-05-10 08:45:57.578528 7f6238602700  0 log [ERR] : 2.5 osd.1 missing a2f462f5/benchmark_data_plana20_7011_object4279/head//2
2013-05-10 08:45:57.734204 7f6238602700  0 log [ERR] : 2.5 osd.1 missing ac8100d/benchmark_data_plana20_7011_object4308/head//2
2013-05-10 08:45:57.794667 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 80cd250d/benchmark_data_plana20_7011_object3939/head//2
2013-05-10 08:45:57.927597 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 4f372a0d/benchmark_data_plana20_7011_object4246/head//2
2013-05-10 08:45:57.977050 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 9b50fc0d/benchmark_data_plana20_7011_object4034/head//2
2013-05-10 08:45:58.150137 7f6238602700  0 log [ERR] : 2.5 osd.1 missing c9ac033d/benchmark_data_plana20_7011_object4177/head//2
2013-05-10 08:45:58.218137 7f6238602700  0 log [ERR] : 2.5 osd.1 missing a337804d/benchmark_data_plana20_7011_object4215/head//2
2013-05-10 08:45:58.238004 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 6835824d/benchmark_data_plana20_7011_object4172/head//2
2013-05-10 08:45:58.368692 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 86628b5d/benchmark_data_plana20_7011_object4307/head//2
2013-05-10 08:45:58.684091 7f6238602700  0 log [ERR] : 2.5 osd.1 missing cf51167d/benchmark_data_plana20_7011_object4229/head//2
2013-05-10 08:45:58.780493 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 6034118d/benchmark_data_plana20_7011_object4102/head//2
2013-05-10 08:45:58.780496 7f6238602700  0 log [ERR] : 2.5 osd.1 missing b1dba38d/benchmark_data_plana20_7011_object4239/head//2
2013-05-10 08:45:58.800962 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 76a3c68d/benchmark_data_plana20_7011_object4068/head//2
2013-05-10 08:45:58.919001 7f6238602700  0 log [ERR] : 2.5 osd.1 missing ac3fb89d/benchmark_data_plana20_7011_object4037/head//2
2013-05-10 08:45:58.967996 7f6238602700  0 log [ERR] : 2.5 osd.1 missing cf90e9d/benchmark_data_plana20_7011_object4321/head//2
2013-05-10 08:45:59.319250 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 7951b0dd/benchmark_data_plana20_7011_object3968/head//2
2013-05-10 08:45:59.340885 7f6238602700  0 log [ERR] : 2.5 osd.1 missing e67703dd/benchmark_data_plana20_7011_object4170/head//2
2013-05-10 08:45:59.400895 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 5de731ed/benchmark_data_plana20_7011_object4020/head//2
2013-05-10 08:45:59.420418 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 3e0903ed/benchmark_data_plana20_7011_object4269/head//2
2013-05-10 08:45:59.473032 7f6238602700  0 log [ERR] : 2.5 osd.1 missing bb44d7ed/benchmark_data_plana20_7011_object4192/head//2
2013-05-10 08:45:59.473037 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 631f08ed/benchmark_data_plana20_7011_object4059/head//2
2013-05-10 08:45:59.497681 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 509ddded/benchmark_data_plana20_7011_object4189/head//2
2013-05-10 08:45:59.549972 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 69d157fd/benchmark_data_plana20_7011_object4275/head//2
2013-05-10 08:45:59.549975 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 6dd499fd/benchmark_data_plana20_7011_object4260/head//2
2013-05-10 08:45:59.560790 7f6238602700  0 log [ERR] : 2.5 osd.1 missing 379eeefd/benchmark_data_plana20_7011_object4165/head//2
2013-05-10 08:45:59.560836 7f6238602700  0 log [ERR] : 2.5 deep-scrub stat mismatch, got 710/665 objects, 0/0 clones, 2908160/2723840 bytes.
2013-05-10 08:45:59.560852 7f6238602700  0 log [ERR] : 2.5 deep-scrub 45 missing, 0 inconsistent objects
2013-05-10 08:45:59.560857 7f6238602700  0 log [ERR] : 2.5 deep-scrub 46 errors


Related issues 1 (0 open1 closed)

Has duplicate Ceph - Bug #5057: assertion in DeleteOp::_beginDuplicateSamuel Just05/13/2013

Actions
Actions #1

Updated by Sage Weil almost 11 years ago

  • Priority changed from Normal to Urgent
Actions #2

Updated by Samuel Just almost 11 years ago

  • Status changed from New to 12
  • Assignee set to Samuel Just

Got it reproduced with logs.

Actions #3

Updated by Samuel Just almost 11 years ago

I think I found the issue. wip_5020. Testing.

Actions #4

Updated by Samuel Just almost 11 years ago

  • Status changed from 12 to Pending Backport
Actions #5

Updated by Samuel Just almost 11 years ago

  • Priority changed from Urgent to High
Actions #6

Updated by Samuel Just almost 11 years ago

eaf3abf3f9a7b13b81736aa558c9084a8f07fdbe and
72bf5f4813c273210b5ced7f7793bc1bf813690c are both required.

Actions #7

Updated by Samuel Just almost 11 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF