Project

General

Profile

Actions
Copy link

Bug #1599

closed

osd assert fail (new_tail >= ondisklog.tail)

Added by Josh Durgin over 12 years ago. Updated over 12 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
OSD
Target version:
v0.38
% Done:

0%

Source:
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

From teuthology:~t/log/osd.6.log.gz:

osd/PG.cc: In function 'void PG::trim_ondisklog(ObjectStore::Transaction&)', in thread '0x7f31175e8700'
osd/PG.cc: 2083: FAILED assert(new_tail >= ondisklog.tail)
 ceph version 0.36-156-g81c5f6a (commit:81c5f6ad1dc14999a6ef834a2769dccbc7d97545)
 1: (PG::trim_ondisklog(ObjectStore::Transaction&)+0x528) [0x6f3538]
 2: (PG::trim(ObjectStore::Transaction&, eversion_t)+0x1d5) [0x6f3775]
 3: (OSD::handle_pg_trim(MOSDPGTrim*)+0x982) [0x57e8e2]
 4: (OSD::_dispatch(Message*)+0x71c) [0x594dec]
 5: (OSD::ms_dispatch(Message*)+0x177) [0x596027]
 6: (SimpleMessenger::dispatch_entry()+0xa0a) [0x61470a]
 7: (SimpleMessenger::DispatchThread::entry()+0x2c) [0x4a3c3c]
 8: (Thread::_entry_func(void*)+0x12) [0x60e582]
 9: (()+0x7971) [0x7f312386d971]
 10: (clone()+0x6d) [0x7f31220fd92d]
Actions
Copy link
 #2

Updated by Sage Weil over 12 years ago

can we reproduce with osd logs on?

Actions
Copy link
 #3

Updated by Josh Durgin over 12 years ago

I'm thrashing with osd debugging. This should turn up (as well as other bugs we've seen while thrashing).

Actions
Copy link
 #4

Updated by Josh Durgin over 12 years ago

  • Priority changed from Normal to High

Happened in 151 as well.

Actions
Copy link
 #5

Updated by Sage Weil over 12 years ago

  • Target version changed from v0.37 to v0.38
Actions
Copy link
 #6

Updated by Sage Weil over 12 years ago

  • Translation missing: en.field_position set to 40
Actions
Copy link
 #7

Updated by Josh Durgin over 12 years ago

Finally reproduced this with debugging - logs and pg and osd dump will be in vit:~joshd/thrash_stuck_active3 in a bit - this ran overnight so they're pretty large.

Actions
Copy link
 #8

Updated by Samuel Just over 12 years ago

  • Status changed from New to Resolved

fccd28df371dceffaf6ff7a50422b6a5b1ee126c should take care of it.

Actions
Copy link

Also available in: Atom PDF