Bug #12774
closedceph add new osd on existing host, status keep down
0%
Description
Description of problem:
health cluster have 3 hosts, one mon and one osd per host, cluster status is 248 active + clean, after add a new osd on existing host,cluster status changed 64 active + remapped 184 active + clean, the new osd process is running, but status in cluster is down
Version:
system: ubuntu 14.04 LTS
kernel: 3.13.0-24-generic
ceph: ceph version 0.80.9
How reproduce:
alway
Step to Reproduce:
1. create new osd on existing host manaully
2. sudo ceph -s check status
Actually result:
new osd stauts keep down
Files
Updated by Jeddy Liu over 8 years ago
- File cluster_osd_tree.png cluster_osd_tree.png added
- File osd_3_running.png osd_3_running.png added
2015-08-25 02:20:16.316941 7f3c8e273800 0 ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047), process ceph-osd, pid 2744
2015-08-25 02:20:16.320304 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: FIEMAP ioctl is supported and appears to work
2015-08-25 02:20:16.320312 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-08-25 02:20:16.321242 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-08-25 02:20:16.322797 7f3c8e273800 0 filestore(/var/lib/ceph/osd/ceph-3) limited size xattrs
2015-08-25 02:20:16.324803 7f3c8e273800 0 filestore(/var/lib/ceph/osd/ceph-3) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2015-08-25 02:20:16.324961 7f3c8e273800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2015-08-25 02:20:16.325216 7f3c8e273800 1 journal _open /var/lib/ceph/osd/ceph-3/journal fd 20: 5368709120 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-08-25 02:20:16.325445 7f3c8e273800 1 journal _open /var/lib/ceph/osd/ceph-3/journal fd 20: 5368709120 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-08-25 02:20:16.325752 7f3c8e273800 1 journal close /var/lib/ceph/osd/ceph-3/journal
2015-08-25 02:20:16.331453 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: FIEMAP ioctl is supported and appears to work
2015-08-25 02:20:16.331465 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-08-25 02:20:16.332423 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-08-25 02:20:16.334484 7f3c8e273800 0 filestore(/var/lib/ceph/osd/ceph-3) limited size xattrs
2015-08-25 02:20:16.336010 7f3c8e273800 0 filestore(/var/lib/ceph/osd/ceph-3) mount: WRITEAHEAD journal mode explicitly enabled in conf
2015-08-25 02:20:16.336151 7f3c8e273800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2015-08-25 02:20:16.325216 7f3c8e273800 1 journal _open /var/lib/ceph/osd/ceph-3/journal fd 20: 5368709120 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-08-25 02:20:16.325445 7f3c8e273800 1 journal _open /var/lib/ceph/osd/ceph-3/journal fd 20: 5368709120 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-08-25 02:20:16.325752 7f3c8e273800 1 journal close /var/lib/ceph/osd/ceph-3/journal
2015-08-25 02:20:16.331453 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: FIEMAP ioctl is supported and appears to work
2015-08-25 02:20:16.331465 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-08-25 02:20:16.332423 7f3c8e273800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-3) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-08-25 02:20:16.334484 7f3c8e273800 0 filestore(/var/lib/ceph/osd/ceph-3) limited size xattrs
2015-08-25 02:20:16.336010 7f3c8e273800 0 filestore(/var/lib/ceph/osd/ceph-3) mount: WRITEAHEAD journal mode explicitly enabled in conf
2015-08-25 02:20:16.336151 7f3c8e273800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2015-08-25 02:20:16.336187 7f3c8e273800 1 journal _open /var/lib/ceph/osd/ceph-3/journal fd 21: 5368709120 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-08-25 02:20:16.336465 7f3c8e273800 1 journal _open /var/lib/ceph/osd/ceph-3/journal fd 21: 5368709120 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-08-25 02:20:16.336841 7f3c8e273800 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello
2015-08-25 02:20:16.339739 7f3c8e273800 0 osd.3 221 crush map has features 1107558400, adjusting msgr requires for clients
2015-08-25 02:20:16.339782 7f3c8e273800 0 osd.3 221 crush map has features 1107558400 was 8705, adjusting msgr requires for mons
2015-08-25 02:20:16.339802 7f3c8e273800 0 osd.3 221 crush map has features 1107558400, adjusting msgr requires for osds
2015-08-25 02:20:16.339842 7f3c8e273800 0 osd.3 221 load_pgs
2015-08-25 02:20:16.339924 7f3c8e273800 0 osd.3 221 load_pgs opened 0 pgs
2015-08-25 02:20:16.341571 7f3c8e273800 -1 osd.3 221 set_disk_tp_priority(22) Invalid argument: osd_disk_thread_ioprio_class is but only the following values are allowed: idle, be or rt
2015-08-25 02:20:16.347734 7f3c7e8d1700 0 osd.3 221 ignoring osdmap until we have initialized
2015-08-25 02:20:16.348325 7f3c7e8d1700 0 osd.3 221 ignoring osdmap until we have initialized
2015-08-25 02:20:16.348621 7f3c8e273800 0 osd.3 221 done with init, starting boot process
Updated by Sage Weil over 8 years ago
- Status changed from New to Need More Info
Can you generate a log with 'debug osd = 20' and 'debug ms = 1'?
Updated by Sage Weil about 7 years ago
- Status changed from Need More Info to Can't reproduce
please reopen if this si still a problem