Project

General

Profile

Bug #22039

bluestore: segv during unmount

Added by Patrick Donnelly about 1 month ago. Updated 22 days ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
Start date:
11/03/2017
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
fs
Release:
Needs Doc:
No
Component(RADOS):
BlueStore

Description

2017-11-03T18:59:24.761 INFO:tasks.ceph.osd.1.smithi147.stderr:2017-11-03 18:59:24.745 37470700 -1 received  signal: Terminated from  PID: 25483 task name: /usr/bin/python /bin/daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1  UID: 0
2017-11-03T18:59:24.763 INFO:tasks.ceph.osd.1.smithi147.stderr:2017-11-03 18:59:24.749 37470700 -1 osd.1 24 *** Got signal Terminated ***
2017-11-03T18:59:25.036 INFO:tasks.ceph.osd.1.smithi147.stderr:2017-11-03 18:59:24.992 37470700 -1 osd.1 24 shutdown
2017-11-03T18:59:30.409 INFO:tasks.ceph.osd.1.smithi147.stderr:*** Caught signal (Segmentation fault) **
2017-11-03T18:59:30.409 INFO:tasks.ceph.osd.1.smithi147.stderr: in thread 37470700 thread_name:signal_handler
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: ceph version 13.0.0-2860-g0426813 (04268137129f4794bf9839091512c8bc26c335ba) mimic (dev)
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 1: (()+0xaa0671) [0xba8671]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 2: (()+0xf5e0) [0xc5915e0]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 3: (operator<<(std::ostream&, bluestore_extent_ref_map_t const&)+0x46) [0xadca06]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 4: (operator<<(std::ostream&, bluestore_shared_blob_t const&)+0x72) [0xadebf2]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 5: (operator<<(std::ostream&, BlueStore::SharedBlob const&)+0xba) [0xa4436a]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 6: (BlueStore::SharedBlobSet::dump(CephContext*, int)+0x16e) [0xa4451e]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 7: (BlueStore::_flush_cache()+0x1ae) [0xa71a8e]
2017-11-03T18:59:30.451 INFO:tasks.ceph.osd.1.smithi147.stderr: 8: (BlueStore::umount()+0x1e6) [0xa720c6]
2017-11-03T18:59:30.452 INFO:tasks.ceph.osd.1.smithi147.stderr: 9: (OSD::shutdown()+0x1685) [0x696785]
2017-11-03T18:59:30.452 INFO:tasks.ceph.osd.1.smithi147.stderr: 10: (OSD::handle_signal(int)+0x11f) [0x696f8f]
2017-11-03T18:59:30.452 INFO:tasks.ceph.osd.1.smithi147.stderr: 11: (SignalHandler::entry()+0x1d7) [0xba99d7]
2017-11-03T18:59:30.452 INFO:tasks.ceph.osd.1.smithi147.stderr: 12: (()+0x7e25) [0xc589e25]
2017-11-03T18:59:30.452 INFO:tasks.ceph.osd.1.smithi147.stderr: 13: (clone()+0x6d) [0xd4d034d]
2017-11-03T18:59:30.503 INFO:tasks.ceph.osd.1.smithi147.stderr:2017-11-03 18:59:30.389 37470700 -1 bluestore(/var/lib/ceph/osd/ceph-1) _flush_cache stray shared blobs on 3.1s2_head
2017-11-03T18:59:30.503 INFO:tasks.ceph.osd.1.smithi147.stderr:2017-11-03 18:59:30.455 37470700 -1 *** Caught signal (Segmentation fault) **
2017-11-03T18:59:30.503 INFO:tasks.ceph.osd.1.smithi147.stderr: in thread 37470700 thread_name:signal_handler

From: /ceph/teuthology-archive/pdonnell-2017-11-03_17:59:39-fs-wip-pdonnell-testing-20171102.200208-testing-basic-smithi/1808226/teuthology.log

and: /ceph/teuthology-archive/pdonnell-2017-11-03_17:59:39-fs-wip-pdonnell-testing-20171102.200208-testing-basic-smithi/1808218/teuthology.log


Related issues

Related to RADOS - Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim Duplicate 09/06/2017
Copied to RADOS - Backport #22150: luminous: bluestore: segv during unmount Resolved

History

#1 Updated by Sage Weil about 1 month ago

  • Status changed from New to Need More Info

https://github.com/ceph/ceph/pull/18714 to get better debug output

#2 Updated by Sage Weil about 1 month ago

  • Related to Bug #21259: bluestore: segv in BlueStore::TwoQCache::_trim added

#5 Updated by Patrick Donnelly about 1 month ago

http://pulpito.ceph.com/pdonnell-2017-11-06_22:14:57-fs-wip-pdonnell-testing-20171106.200337-testing-basic-smithi/1821222

OSD crash log: /ceph/teuthology-archive/pdonnell-2017-11-06_22:14:57-fs-wip-pdonnell-testing-20171106.200337-testing-basic-smithi/1821222/remote/smithi136/log/ceph-osd.1.log.gz

#6 Updated by Sage Weil about 1 month ago

  • Status changed from Need More Info to Need Review
  • Priority changed from Normal to Urgent
  • Backport set to luminous

#7 Updated by Sage Weil about 1 month ago

  • Status changed from Need Review to Pending Backport

#8 Updated by Prashant D 29 days ago

  • Copied to Backport #22150: luminous: bluestore: segv during unmount added

#9 Updated by Nathan Cutler 22 days ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF