Project

General

Profile

Bug #11757

Failure to load the osdmap

Added by Irek Fasikhov almost 5 years ago. Updated almost 5 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

Compiling from source the latest active version Hammer.

[root@ceph01p24 ~]# ceph -v
ceph version 0.94.1-116-g63832d4 (63832d4039889b6b704b88b86eaba4aadcfceb2e)
    -5> 2015-05-26 09:42:54.997851 7f0fbfa34880 10 _load_class version success
    -4> 2015-05-26 09:42:54.997862 7f0fbfa34880 20 osd.25 0 get_map 17735 - loading and decoding 0x4589200
    -3> 2015-05-26 09:42:54.997869 7f0fbfa34880 15 filestore(/var/lib/ceph/osd/ceph-25) read meta/4e928679/osdmap.17735/0//-1 0~0
    -2> 2015-05-26 09:42:54.997890 7f0fbfa34880 10 filestore(/var/lib/ceph/osd/ceph-25) error opening file /var/lib/ceph/osd/ceph-25/current/meta/DIR_9/DIR_7/osdmap.17735__0_4E928679__none with flags=2: (2) No such file or directory
    -1> 2015-05-26 09:42:54.997899 7f0fbfa34880 10 filestore(/var/lib/ceph/osd/ceph-25) FileStore::read(meta/4e928679/osdmap.17735/0//-1) open error: (2) No such file or directory
     0> 2015-05-26 09:42:54.999254 7f0fbfa34880 -1 osd/OSD.h: In function 'OSDMapRef OSDService::get_map(epoch_t)' thread 7f0fbfa34880 time 2015-05-26 09:42:54.997908
osd/OSD.h: 716: FAILED assert(ret)

 ceph version 0.94.1-116-g63832d4 (63832d4039889b6b704b88b86eaba4aadcfceb2e)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0xbc4e15]
 2: (OSDService::get_map(unsigned int)+0x3f) [0x6ffa9f]
 3: (OSD::init()+0x6b7) [0x6b8e17]
 4: (main()+0x27f3) [0x643b63]
 5: (__libc_start_main()+0xf5) [0x7f0fbcdd2af5]
 6: /usr/bin/ceph-osd() [0x65cdc9]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Configuration:

[osd]
        osd journal size = 10000
        osd mkfs type = xfs
        osd mkfs options xfs = -f -i size=2048
        osd mount options xfs = rw,noatime,inode64,logbsize=256k,allocsize=1m
        filestore xattr use omap = true

        osd scrub load threshold = 2
        osd recovery op priority = 2
        osd max backfills = 1
        osd recovery max active = 1
        osd recovery threads = 1
        osd crush update on start = false
        osd recovery delay start = 5
        osd snap trim sleep = 0.5
        osd disk thread ioprio class = idle
        osd disk thread ioprio priority = 7

        debug_objecter = 20/20
        debug_ms = 20/20
        debug_filestore = 20/20
        debug_osd = 20/20
        debug_journal = 20/20

System:

[root@ceph01p24 ~]# uname -a
Linux ceph01p24.bank-hlynov.ru 3.10.0-229.4.2.el7.x86_64 #1 SMP Wed May 13 10:06:09 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@ceph01p24 ~]# cat /etc/redhat-release 
CentOS Linux release 7.1.1503 (Core)

ceph-osd.25.log.tar.gz (173 KB) Irek Fasikhov, 05/26/2015 09:32 AM


Related issues

Duplicates Ceph - Bug #11680: mon crashes when "ceph osd tree 85 --format json" Can't reproduce 05/19/2015

History

#1 Updated by Loic Dachary almost 5 years ago

  • Priority changed from Immediate to Normal

it would be useful to know more about the history of this OSD. Was it upgraded from a previous version ? Is it a fresh install ? Did it experience problems in the past ? Just to rule out the obvious (i.e. the osdmap was removed because the disk had a catastrophic failure etc.).

#2 Updated by Irek Fasikhov almost 5 years ago

Loic Dachary wrote:

it would be useful to know more about the history of this OSD. Was it upgraded from a previous version ? Is it a fresh install ? Did it experience problems in the past ? Just to rule out the obvious (i.e. the osdmap was removed because the disk had a catastrophic failure etc.).

It was a fresh installation. Multiple server disconnects the power supply, but there is no error in the FS. And these OSD 5 pieces.

[root@ceph01p24 ~]# ceph osd tree | grep down
 84   0.79999         osd.84        down        0          1.00000 
101   0.79999         osd.101       down        0          1.00000 
 25   3.62999         osd.25        down        0          1.00000 
 31   3.62999         osd.31        down        0          1.00000 
 62   3.62999         osd.62        down        0          1.00000 

#3 Updated by Loic Dachary almost 5 years ago

Ok. Could you add to this issue as much details as possible regarding how this new OSD (osd.25) was installed ? Maybe the problem can be reproduced by following your instructions ?

#4 Updated by Loic Dachary almost 5 years ago

Multiple server disconnects the power supply, but there is no error in the FS

Do you mean the server on which OSD.25 has been installed had a power failure, after or during the installation of the OSD ?

#5 Updated by Loic Dachary almost 5 years ago

  • Status changed from New to Duplicate

Duplicates #11680

Also available in: Atom PDF