Project

General

Profile

Bug #23029

osd does not handle eio on meta objects (e.g., osdmap)

Added by Sage Weil over 1 year ago. Updated over 1 year ago.

Status:
Verified
Priority:
High
Assignee:
-
Category:
Correctness/Safety
Target version:
-
Start date:
02/16/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:

Description

2018-02-16 18:12:41.377 7f3702bb0700 15 bluestore(/var/lib/ceph/osd/ceph-1) read meta #-1:3d48e826:::osdmap.38:0# 0x0~0
2018-02-16 18:12:41.377 7f3702bb0700 20 bluestore(/var/lib/ceph/osd/ceph-1) _do_read 0x0~bf2 size 0xbf2 (3058)
2018-02-16 18:12:41.377 7f3702bb0700 20 bluestore(/var/lib/ceph/osd/ceph-1) _do_read will do buffered read
2018-02-16 18:12:41.377 7f3702bb0700 20 bluestore(/var/lib/ceph/osd/ceph-1) _do_read  blob Blob(0x5636408dd330 blob([0x112df4000~4000] csum+has_unused crc32c/0x1000 unused=0xfff0) use_tracker(0x4000 0xbf2) SharedBlob(0x56363d7baa10 sbid 0x0)) need 0x0~bf2 cache has 0x[0~bf2]
2018-02-16 18:12:41.377 7f3702bb0700  0 bluestore(/var/lib/ceph/osd/ceph-1) read: inject random EIO
2018-02-16 18:12:41.377 7f3702bb0700 10 bluestore(/var/lib/ceph/osd/ceph-1) read meta #-1:3d48e826:::osdmap.38:0# 0x0~bf2 = -5
2018-02-16 18:12:41.377 7f3702bb0700 -1 /build/ceph-13.0.1-2002-gf4bbaff/src/osd/OSD.cc: In function 'void OSD::handle_osd_map(MOSDMap*)' thread 7f3702bb0700 time 2018-02-16 18:12:41.381004
/build/ceph-13.0.1-2002-gf4bbaff/src/osd/OSD.cc: 7223: FAILED assert(p != added_maps_bl.end())

 ceph version 13.0.1-2002-gf4bbaff (f4bbaff64800dbf406deab3e41328d6a15498fce) mimic (dev)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xf5) [0x7f3714caae95]
 2: (OSD::handle_osd_map(MOSDMap*)+0x22b2) [0x56363a5cd2b2]
 3: (OSD::_dispatch(Message*)+0xed) [0x56363a5d524d]
 4: (OSD::ms_dispatch(Message*)+0x67) [0x56363a5d56e7]
 5: (DispatchQueue::entry()+0xc6a) [0x7f3714d2877a]
 6: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f3714dc3d8d]
 7: (()+0x76ba) [0x7f371377f6ba]
 8: (clone()+0x6d) [0x7f3712fa83dd]

/a/sage-2018-02-16_16:11:07-rados-wip-sage-testing-2018-02-16-0837-distro-basic-smithi/2193561

History

#1 Updated by Josh Durgin over 1 year ago

  • Project changed from Ceph to RADOS
  • Category set to Correctness/Safety

We could at least fail more politely here even if we can't recover from it in the short term.

Also available in: Atom PDF