Project

General

Profile

Bug #13477

crush: crash if we see CRUSH_ITEM_NONE in early rule step

Added by Sage Weil almost 7 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
infernalis, hammer
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

see bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=1231630

Description of problem:
Seeing Monitor Crash while creating erasure coded pool with wrong parameters

Version-Release number of selected component (if applicable):
ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)

How reproducible:
1/1

Steps to Reproduce:
1. Create a ec profile, with the below command.

ceph osd erasure-code-profile set myprofile plugin=lrc mapping=__DD__DD layers='[[ "_cDD_cDD", "" ],[ "cDDD____", "" ],[ "____cDDD", "" ],]' ruleset-steps='[ [ "choose", "datacenter", 3 ], [ "chooseleaf", "osd", 0] ]'

2. ceph osd pool create ecpool 12 12 erasure myprofile

Actual results: Monitoring is crashing.
BT:

ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)
 1: /usr/bin/ceph-mon() [0x901e52]
 2: (()+0xf130) [0x7f205a9ca130]
 3: (crush_do_rule()+0x291) [0x833501]
 4: (OSDMap::_pg_to_osds(pg_pool_t const&, pg_t, std::vector<int, std::allocator<int> >, int, unsigned int*) const+0xff) [0x77b76f]
 5: (OSDMap::_pg_to_up_acting_osds(pg_t const&, std::vector<int, std::allocator<int> >, int, std::vector<int, std::allocator<int> >, int) const+0x104) [0x77be24]
 6: (PGMonitor::map_pg_creates()+0x268) [0x65b748]
 7: (PGMonitor::post_paxos_update()+0x25) [0x65bf35]
 8: (Monitor::refresh_from_paxos(bool*)+0x221) [0x575721]
 9: (Monitor::init_paxos()+0x95) [0x575ac5]
 10: (Monitor::preinit()+0x7f1) [0x57a881]
 11: (main()+0x24a1) [0x54d881]
 12: (__libc_start_main()+0xf5) [0x7f20593d0af5]
 13: /usr/bin/ceph-mon() [0x55d0f9]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

cm (589 Bytes) Sage Weil, 10/13/2015 01:47 PM

Related issues

Copied to Ceph - Backport #13654: crush: crash if we see CRUSH_ITEM_NONE in early rule step Resolved
Copied to Ceph - Backport #13655: crush: crash if we see CRUSH_ITEM_NONE in early rule step Resolved

Associated revisions

Revision 976a24a3 (diff)
Added by Sage Weil almost 7 years ago

crush/mapper: ensure bucket id is valid before indexing buckets array

We were indexing the buckets array without verifying the index was within
the [0,max_buckets) range. This could happen because a multistep rule
does not have enough buckets and has CRUSH_ITEM_NONE
for an intermediate result, which would feed in CRUSH_ITEM_NONE and
make us crash.

Fixes: #13477
Signed-off-by: Sage Weil <>

Revision 81d8aa14 (diff)
Added by Sage Weil almost 7 years ago

crush/mapper: ensure bucket id is valid before indexing buckets array

We were indexing the buckets array without verifying the index was within
the [0,max_buckets) range. This could happen because a multistep rule
does not have enough buckets and has CRUSH_ITEM_NONE
for an intermediate result, which would feed in CRUSH_ITEM_NONE and
make us crash.

Fixes: #13477
Signed-off-by: Sage Weil <>
(cherry picked from commit 976a24a326da8931e689ee22fce35feab5b67b76)

Revision ecb6aa23 (diff)
Added by Sage Weil almost 7 years ago

crush/mapper: ensure bucket id is valid before indexing buckets array

We were indexing the buckets array without verifying the index was within
the [0,max_buckets) range. This could happen because a multistep rule
does not have enough buckets and has CRUSH_ITEM_NONE
for an intermediate result, which would feed in CRUSH_ITEM_NONE and
make us crash.

Fixes: #13477
Signed-off-by: Sage Weil <>
(cherry picked from commit 976a24a326da8931e689ee22fce35feab5b67b76)

History

#1 Updated by Sage Weil almost 7 years ago

  • Backport changed from hammer to infernalis, hammer

#2 Updated by Sage Weil almost 7 years ago

Good news is that latest code prevents the rule from being created in the first place (yay crushtool check). But crush shouldn't crash.

#3 Updated by Sage Weil almost 7 years ago

  • File cm added

sample crush map attached

#4 Updated by Sage Weil almost 7 years ago

  • Status changed from 12 to Fix Under Review

#5 Updated by Loïc Dachary almost 7 years ago

  • Backport changed from infernalis, hammer to infernalis, hammer, firefly

#6 Updated by Loïc Dachary almost 7 years ago

  • Assignee set to Sage Weil

#7 Updated by Sage Weil almost 7 years ago

  • Status changed from Fix Under Review to Pending Backport
  • Assignee deleted (Sage Weil)

#8 Updated by Nathan Cutler almost 7 years ago

  • Copied to Backport #13653: crush: crash if we see CRUSH_ITEM_NONE in early rule step added

#9 Updated by Nathan Cutler almost 7 years ago

  • Copied to Backport #13654: crush: crash if we see CRUSH_ITEM_NONE in early rule step added

#10 Updated by Nathan Cutler almost 7 years ago

  • Copied to Backport #13655: crush: crash if we see CRUSH_ITEM_NONE in early rule step added

#11 Updated by Loïc Dachary over 6 years ago

  • Backport changed from infernalis, hammer, firefly to infernalis, hammer

#12 Updated by Loïc Dachary over 6 years ago

  • Copied to deleted (Backport #13653: crush: crash if we see CRUSH_ITEM_NONE in early rule step)

#13 Updated by Loïc Dachary over 6 years ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF