Bug #20092: ceph-osd: FileStore::_do_transaction: assert(0 == "unexpected error") - RADOS - Ceph

Actions

Copy link

Bug #20092

closed

ceph-osd: FileStore::_do_transaction: assert(0 == "unexpected error")

Added by Jason Dillaman almost 7 years ago. Updated almost 7 years ago.

Status:

Duplicate

Priority:

Normal

Assignee:

Jason Dillaman

Category:

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(RADOS):

FileStore

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

http://pulpito.ceph.com/jdillaman-2017-05-25_16:48:38-rbd-wip-jd-testing-distro-basic-smithi/1229611

#1  0x00007fb71445f8c8 in abort () from /lib64/libc.so.6
#2  0x00007fb7189e6b14 in ceph::__ceph_assert_fail (assertion=assertion@entry=0x7fb718e454a2 "0 == \"unexpected error\"", 
    file=file@entry=0x7fb718e45920 "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.2-1612-gec739374/rpm/el7/BUILD/ceph-12.0.2-1"..., 
    line=line@entry=3097, 
    func=func@entry=0x7fb718e49260 <FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int, ThreadPool::TPHandle*)::__PRETTY_FUNCTION__> "void FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t, int, ThreadPool::TPHandle*)")
    at /usr/src/debug/ceph-12.0.2-1612-gec739374/src/common/assert.cc:77
#3  0x00007fb7187d6afa in FileStore::_do_transaction (this=this@entry=0x7fb723356000, t=..., op_seq=op_seq@entry=435858, 
    trans_num=trans_num@entry=0, handle=handle@entry=0x7fb70cd997d0)
    at /usr/src/debug/ceph-12.0.2-1612-gec739374/src/os/filestore/FileStore.cc:3097
#4  0x00007fb7187dc24b in FileStore::_do_transactions (this=this@entry=0x7fb723356000, 
    tls=std::vector of length 2, capacity 2 = {...}, op_seq=435858, handle=handle@entry=0x7fb70cd997d0)
    at /usr/src/debug/ceph-12.0.2-1612-gec739374/src/os/filestore/FileStore.cc:2274
#5  0x00007fb7187dc5db in FileStore::_do_op (this=0x7fb723356000, osr=0x7fb7245782c0, handle=...)
    at /usr/src/debug/ceph-12.0.2-1612-gec739374/src/os/filestore/FileStore.cc:2017
#6  0x00007fb7189ed7d9 in ThreadPool::worker (this=0x7fb723356b00, wt=0x7fb7235b3ec0)
    at /usr/src/debug/ceph-12.0.2-1612-gec739374/src/common/WorkQueue.cc:122
#7  0x00007fb7189ee7b0 in ThreadPool::WorkThread::entry (this=<optimized out>)
    at /usr/src/debug/ceph-12.0.2-1612-gec739374/src/common/WorkQueue.h:443
#8  0x00007fb715631dc5 in start_thread () from /lib64/libpthread.so.0
#9  0x00007fb71452073d in clone () from /lib64/libc.so.6

Actions

Copy link

Updated by Greg Farnum almost 7 years ago

Project changed from Ceph to RADOS
Assignee set to Jason Dillaman
Priority changed from Urgent to Normal
Component(RADOS) FileStore added

Do you have any evidence this wasn't an unexpected error given to us by the Filesystems, Jason? That does happen in the lab...

Actions

Copy link

Updated by Jason Dillaman almost 7 years ago

Status changed from New to Rejected

The osd.1 log showed the rocksdb encountered a full disk:

-17> 2017-05-25 22:14:28.664403 7fb70cd9b700 -1 rocksdb: submit_transaction error: IO error: No space left on device/var/lib/ceph
/osd/cluster1-1/current/omap/000017.sst: No space left on device code = 5 Rocksdb transaction:

Actions

Copy link