Project

General

Profile

Actions

Bug #8891

closed

rados bench hang during thrashing

Added by Sage Weil almost 10 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

teuthology-2014-07-20_02:30:01-rados-next-testing-basic-plana/371201

we thrash and recover but the rados bench gets stuck. it gets stuck almost immediately:

2014-07-20T10:43:58.204 INFO:teuthology.task.radosbench.radosbench.0.out:[10.214.132.36]:     70      15       634       619   35.3653         0         -    1.0567

unfortunately not much in the way of logs


Related issues 1 (0 open1 closed)

Has duplicate Ceph - Bug #8939: stalled LibRadosTwoPoolsPP.TryFlushReadRace; client failed to reconnect?DuplicateSage Weil07/26/2014

Actions
Actions #1

Updated by Tamilarasi muthamizhan almost 10 years ago

  • Assignee set to Tamilarasi muthamizhan
Actions #2

Updated by Tamilarasi muthamizhan almost 10 years ago

sage: also, issue 8891, adjust the nightly qa so that logging is enabled for the rados bench run.  i think that means modifying radosbench.py to pass --debug-ms 1, --debug-objecter 20, debug rados = 20
Actions #3

Updated by Tamilarasi muthamizhan over 9 years ago

  • Status changed from 12 to Resolved

added debug messages to radosbench.yaml

commit 367d4da083ea47b1de9201bbda943e57617f6701

also cherry-picked to ceph-qa-suite next branch.

Actions #4

Updated by Sage Weil over 9 years ago

  • Status changed from Resolved to Need More Info
  • Assignee deleted (Tamilarasi muthamizhan)
  • Priority changed from Urgent to High

now that the logging is there we wait for it to happen again...

Actions #5

Updated by Sage Weil over 9 years ago

  • Assignee set to Sage Weil
Actions #6

Updated by Sage Weil over 9 years ago

  • Status changed from Need More Info to 7
  • Priority changed from High to Urgent

i think this was the same repaer vs fast dispatch that i tracked down in wip-msgr.

Actions #7

Updated by Sage Weil over 9 years ago

  • Status changed from 7 to Fix Under Review
Actions #8

Updated by Sage Weil over 9 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF