Project

General

Profile

Bug #1599

osd assert fail (new_tail >= ondisklog.tail)

Added by Josh Durgin almost 9 years ago. Updated almost 9 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
OSD
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

From teuthology:~/log/osd.6.log.gz:

osd/PG.cc: In function 'void PG::trim_ondisklog(ObjectStore::Transaction&)', in thread '0x7f31175e8700'
osd/PG.cc: 2083: FAILED assert(new_tail >= ondisklog.tail)
 ceph version 0.36-156-g81c5f6a (commit:81c5f6ad1dc14999a6ef834a2769dccbc7d97545)
 1: (PG::trim_ondisklog(ObjectStore::Transaction&)+0x528) [0x6f3538]
 2: (PG::trim(ObjectStore::Transaction&, eversion_t)+0x1d5) [0x6f3775]
 3: (OSD::handle_pg_trim(MOSDPGTrim*)+0x982) [0x57e8e2]
 4: (OSD::_dispatch(Message*)+0x71c) [0x594dec]
 5: (OSD::ms_dispatch(Message*)+0x177) [0x596027]
 6: (SimpleMessenger::dispatch_entry()+0xa0a) [0x61470a]
 7: (SimpleMessenger::DispatchThread::entry()+0x2c) [0x4a3c3c]
 8: (Thread::_entry_func(void*)+0x12) [0x60e582]
 9: (()+0x7971) [0x7f312386d971]
 10: (clone()+0x6d) [0x7f31220fd92d]

History

#2 Updated by Sage Weil almost 9 years ago

can we reproduce with osd logs on?

#3 Updated by Josh Durgin almost 9 years ago

I'm thrashing with osd debugging. This should turn up (as well as other bugs we've seen while thrashing).

#4 Updated by Josh Durgin almost 9 years ago

  • Priority changed from Normal to High

Happened in 151 as well.

#5 Updated by Sage Weil almost 9 years ago

  • Target version changed from v0.37 to v0.38

#6 Updated by Sage Weil almost 9 years ago

  • translation missing: en.field_position set to 40

#7 Updated by Josh Durgin almost 9 years ago

Finally reproduced this with debugging - logs and pg and osd dump will be in vit:~joshd/thrash_stuck_active3 in a bit - this ran overnight so they're pretty large.

#8 Updated by Samuel Just almost 9 years ago

  • Status changed from New to Resolved

fccd28df371dceffaf6ff7a50422b6a5b1ee126c should take care of it.

Also available in: Atom PDF