Project

General

Profile

Bug #20704

osd/PGLog.h: 1204: FAILED assert(missing.may_include_deletes)

Added by Sage Weil 5 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Immediate
Assignee:
Category:
-
Target version:
-
Start date:
07/20/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Release:
Needs Doc:
No
Component(RADOS):

Description

2017-07-20T06:11:56.105 INFO:tasks.ceph.osd.3.smithi055.stderr:2017-07-20 06:11:56.105767 7f08546c1700 -1 received  signal: Hangup from  PID: 18366 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3  UID: 0
2017-07-20T06:11:56.189 INFO:tasks.ceph.osd.4.smithi055.stderr:/build/ceph-12.1.1-195-gb80122d/src/osd/PGLog.h: In function 'static void PGLog::read_log_and_missing(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, PGLog::IndexedLog&, missing_type&, std::ostringstream&, bool, bool*, const DoutPrefixProvid
er*, std::set<std::basic_string<char> >*, bool) [with missing_type = pg_missing_set<true>; std::ostringstream = std::basic_ostringstream<char>]' thread 7fa4496a8b80 time 2017-07-20 06:11:56.189349
2017-07-20T06:11:56.189 INFO:tasks.ceph.osd.4.smithi055.stderr:/build/ceph-12.1.1-195-gb80122d/src/osd/PGLog.h: 1204: FAILED assert(missing.may_include_deletes)
2017-07-20T06:11:56.190 INFO:teuthology.orchestra.run.smithi055:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight'
2017-07-20T06:11:56.214 INFO:tasks.ceph.osd.2.smithi011.stderr:2017-07-20 06:11:56.206049 7f0e05ec6700 -1 received  signal: Hangup from  PID: 313189 task name: /usr/bin/python /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2  UID: 0
2017-07-20T06:11:56.249 INFO:tasks.ceph.osd.4.smithi055.stderr: ceph version 12.1.1-195-gb80122d (b80122de4918e6fc0df376627bde328f84d50be1) luminous (rc)
2017-07-20T06:11:56.249 INFO:tasks.ceph.osd.4.smithi055.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x7fa448dab08e]
2017-07-20T06:11:56.249 INFO:tasks.ceph.osd.4.smithi055.stderr: 2: (void PGLog::read_log_and_missing<pg_missing_set<true> >(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, PGLog::IndexedLog&, pg_missing_set<true>&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, bool, bool*, DoutPrefixProvider const*, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*, bool)+0x24d2) [0x7fa44895c782]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4de) [0x7fa44891651e]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: 4: (OSD::load_pgs()+0x98c) [0x7fa44886b87c]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: 5: (OSD::init()+0x1e74) [0x7fa448881f64]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: 6: (main()+0x29f6) [0x7fa44878ea66]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: 7: (__libc_start_main()+0xf5) [0x7fa4458a0f45]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: 8: (()+0x49f266) [0x7fa448829266]
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr:2017-07-20 06:11:56.248321 7fa4496a8b80 -1 /build/ceph-12.1.1-195-gb80122d/src/osd/PGLog.h: In function 'static void PGLog::read_log_and_missing(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, PGLog::IndexedLog&, missing_type&, std::ostringstream&, bool, bool*, const DoutPrefixProvider*, std::set<std::basic_string<char> >*, bool) [with missing_type = pg_missing_set<true>; std::ostringstream = std::basic_ostringstream<char>]' thread 7fa4496a8b80 time 2017-07-20 06:11:56.189349
2017-07-20T06:11:56.250 INFO:tasks.ceph.osd.4.smithi055.stderr:/build/ceph-12.1.1-195-gb80122d/src/osd/PGLog.h: 1204: FAILED assert(missing.may_include_deletes)

/a/sage-2017-07-20_05:37:32-rados-wip-sage-testing2-distro-basic-smithi/1424027

History

#1 Updated by Josh Durgin 5 months ago

  • Status changed from Verified to In Progress
  • Assignee set to Josh Durgin

Found a bug in my ceph-objectstore-tool change that could cause this, seeing if it did in this case.

#2 Updated by Josh Durgin 5 months ago

This was a bug in persisting the missing state during split. Building a fix.

#3 Updated by Josh Durgin 5 months ago

  • Status changed from In Progress to Need Review

#4 Updated by Sage Weil 5 months ago

  • Status changed from Need Review to Resolved

Also available in: Atom PDF