Project

General

Profile

Actions
Copy link

Backport #14292

closed

osd/PG.cc: 3837: FAILED assert(0 == "Running incompatible OSD")

Added by Yuri Weinstein over 8 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
David Zafman
Target version:
v0.94.6
Release:
hammer
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

https://github.com/ceph/ceph/pull/7206

Related issues 1 (0 open — 1 closed)

Copied from Ceph - Bug #11661: "FAILED assert(0 == "Running incompatible OSD")"ResolvedDavid Zafman05/16/2015

Actions
Actions
Copy link
 #1

Updated by Tamilarasi muthamizhan over 8 years ago

latest log: @teuthology.ovh.sepia.ceph.com: /a/teuthology-2016-01-06_20:55:01-rados-hammer-distro-basic-openstack/61025

Actions
Copy link
 #2

Updated by Samuel Just over 8 years ago

  • Assignee set to David Zafman
  • Priority changed from Normal to Urgent
Actions
Copy link
 #3

Updated by David Zafman over 8 years ago

osd.4 asserted that its connection to osd.1 didn't have the feature bit 19 CEPH_FEATURE_CHUNKY_SCRUB set. Since all nodes are running 0.94.5-178-g9739d4d that feature bit should be set. 

-114> 2016-01-06 21:57:40.561263 7f8a7fab6700 20 osd.4 pg_epoch: 19 pg[0.6( empty local-les=10 n=0 ec=1 les/c 10/10 9/9/9) [4,1] r=0 lpr=9 crt=0'0 mlcod 0'0 active+clean+scrubbing] OSD 1 does not support chunky scrubs, falling back to classic

As expected a previous message on osd.4 shows that osd.1 was reporting that feature through a different mechanism:

2016-01-06 21:50:35.216789 7f8a862c3700 20 osd.4 pg_epoch: 16 pg[1.3( empty local-les=0 n=0 ec=5 les/c 0/5 9/9/9) [4,1] r=0 lpr=16 pi=5-8/2 crt=0'0 mlcod 0'0 creating+peering] state<Started/Primary/Peering/GetInfo>: Adding osd: 1 features: 83ffffffffffff

Actions
Copy link
 #4

Updated by David Zafman over 8 years ago

  • Is duplicate of Bug #11661: "FAILED assert(0 == "Running incompatible OSD")" added
Actions
Copy link
 #5

Updated by David Zafman over 8 years ago

  • Status changed from New to 17
  • Target version set to v0.94.6
Actions
Copy link
 #6

Updated by David Zafman over 8 years ago

Pull request #7206 "Backports to fix 14292"

Actions
Copy link
 #7

Updated by Loïc Dachary over 8 years ago

  • Tracker changed from Bug to Backport
  • Description updated (diff)
  • Status changed from 17 to In Progress
  • Target version deleted (v0.94.6)

Original description

Run: http://pulpito.ovh.sepia.ceph.com:8081/teuthology-2016-01-06_20:55:01-rados-hammer-distro-basic-openstack/
Job: ['61025']
Logs: http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-01-06_20:55:01-rados-hammer-distro-basic-openstack/61025/teuthology.log

2016-01-06T21:57:40.672 INFO:tasks.ceph.osd.4.target082085.stderr:osd/PG.cc: In function 'void PG::scrub(ThreadPool::TPHandle&)' thread 7f8a7fab6700 time 2016-01-06 21:57:40.561289
2016-01-06T21:57:40.672 INFO:tasks.ceph.osd.4.target082085.stderr:osd/PG.cc: 3837: FAILED assert(0 == "Running incompatible OSD")
2016-01-06T21:57:40.674 INFO:tasks.ceph.osd.4.target082085.stderr: ceph version 0.94.5-178-g9739d4d (9739d4de49f8167866eda556b2f1581c068ec8a7)
2016-01-06T21:57:40.674 INFO:tasks.ceph.osd.4.target082085.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0xbc5c7b]
2016-01-06T21:57:40.674 INFO:tasks.ceph.osd.4.target082085.stderr: 2: (PG::scrub(ThreadPool::TPHandle&)+0x5fb) [0x7fbbfb]
2016-01-06T21:57:40.675 INFO:tasks.ceph.osd.4.target082085.stderr: 3: (OSD::ScrubWQ::_process(PG*, ThreadPool::TPHandle&)+0x19) [0x6cdc79]
2016-01-06T21:57:40.675 INFO:tasks.ceph.osd.4.target082085.stderr: 4: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa56) [0xbb6866]
2016-01-06T21:57:40.675 INFO:tasks.ceph.osd.4.target082085.stderr: 5: (ThreadPool::WorkThread::entry()+0x10) [0xbb7910]
2016-01-06T21:57:40.675 INFO:tasks.ceph.osd.4.target082085.stderr: 6: (()+0x8182) [0x7f8aa0b72182]
2016-01-06T21:57:40.676 INFO:tasks.ceph.osd.4.target082085.stderr: 7: (clone()+0x6d) [0x7f8a9f0dd47d]
2016-01-06T21:57:40.676 INFO:tasks.ceph.osd.4.target082085.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this
Actions
Copy link
 #8

Updated by Loïc Dachary over 8 years ago

  • Is duplicate of deleted (Bug #11661: "FAILED assert(0 == "Running incompatible OSD")")
Actions
Copy link
 #9

Updated by Loïc Dachary over 8 years ago

  • Copied from Bug #11661: "FAILED assert(0 == "Running incompatible OSD")" added
Actions
Copy link
 #10

Updated by Loïc Dachary over 8 years ago

  • Subject changed from "osd/PG.cc: 3837: FAILED assert(0 == "Running incompatible OSD")" in rados-hammer-distro-basic-openstack to "osd/PG.cc: 3837: FAILED assert(0 == "Running incompatible OSD")"
Actions
Copy link
 #11

Updated by Loïc Dachary over 8 years ago

  • Subject changed from "osd/PG.cc: 3837: FAILED assert(0 == "Running incompatible OSD")" to osd/PG.cc: 3837: FAILED assert(0 == "Running incompatible OSD")
Actions
Copy link
 #12

Updated by Sage Weil about 8 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #13

Updated by Loïc Dachary over 7 years ago

  • Target version set to v0.94.6
Actions
Copy link

Also available in: Atom PDF