Ceph

The problem did not show when run again, it is a race condition.

tried to

    kill -STOP $(cat $dir/osd-$id.pidfile)

The only place where an osd will return ENXIO is when a request is OSDService::handle_misdirected_op . However, since there only is a single OSD, it can hardly be confused with another.

failure log

 TEST_config_track: 55: run_osd osd-config 0
run_osd: 19: local dir=osd-config
run_osd: 20: shift
run_osd: 21: local id=0
run_osd: 22: shift
run_osd: 23: local osd_data=osd-config/0
run_osd: 25: local ceph_disk_args
run_osd: 26: ceph_disk_args+=' --statedir=osd-config'
run_osd: 27: ceph_disk_args+=' --sysconfdir=osd-config'
run_osd: 28: ceph_disk_args+=' --prepend-to-path='
run_osd: 29: ceph_disk_args+=' --verbose'
run_osd: 31: touch osd-config/ceph.conf
run_osd: 33: ./ceph-disk --statedir=osd-config --sysconfdir=osd-config --prepend-to-path= --verbose prepare osd-config/0
DEBUG:ceph-disk:Preparing osd data dir osd-config/0
run_osd: 36: local 'ceph_args=--fsid=b52a3ef3-fedb-4553-b888-725a94c25bc5 --auth-supported=none --mon-host=127.0.0.1 '
run_osd: 37: ceph_args+=' --osd-journal-size=100'
run_osd: 38: ceph_args+=' --osd-data=osd-config/0'
run_osd: 39: ceph_args+=' --chdir='
run_osd: 40: ceph_args+=' --run-dir=osd-config'
run_osd: 41: ceph_args+=' --debug-osd=20'
run_osd: 42: ceph_args+=' --log-file=osd-config/osd-$id.log'
run_osd: 43: ceph_args+=' --pid-file=osd-config/osd-$id.pidfile'
run_osd: 44: ceph_args+=' '
run_osd: 45: ceph_args+=
run_osd: 46: CEPH_ARGS='--fsid=b52a3ef3-fedb-4553-b888-725a94c25bc5 --auth-supported=none --mon-host=127.0.0.1 --osd-journal-size=100 --osd-data=osd-config/0 --chdir= --run-dir=osd-config --debug-osd=20 --log-file=osd-config/osd-$id.log --pid-file=osd-config/osd-$id.pidfile '
run_osd: 46: ./ceph-disk --statedir=osd-config --sysconfdir=osd-config --prepend-to-path= --verbose activate --mark-init=none osd-config/0
DEBUG:ceph-disk:Cluster uuid is b52a3ef3-fedb-4553-b888-725a94c25bc5
DEBUG:ceph-disk:Cluster name is ceph
DEBUG:ceph-disk:OSD uuid is ea741a0b-35b5-45e4-af2c-406029e5a8cd
DEBUG:ceph-disk:Allocating OSD id...
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
DEBUG:ceph-disk:OSD id is 0
DEBUG:ceph-disk:Initializing OSD...
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
got monmap epoch 1
2014-06-25 15:11:39.528729 2b974e271ac0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2014-06-25 15:11:41.103587 2b974e271ac0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2014-06-25 15:11:41.111786 2b974e271ac0 -1 filestore(osd-config/0) could not find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory
2014-06-25 15:11:41.204046 2b974e271ac0 -1 created object store osd-config/0 journal osd-config/0/journal for osd.0 fsid b52a3ef3-fedb-4553-b888-725a94c25bc5
2014-06-25 15:11:41.204113 2b974e271ac0 -1 auth: error reading file: osd-config/0/keyring: can't open osd-config/0/keyring: (2) No such file or directory
2014-06-25 15:11:41.204222 2b974e271ac0 -1 created new key in keyring osd-config/0/keyring
DEBUG:ceph-disk:Authorizing OSD key...
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
added key for osd.0
DEBUG:ceph-disk:ceph osd.0 data dir is ready at osd-config/0
starting osd.0 at :/0 osd_data osd-config/0 osd-config/0/journal
rrun_osd: 51: cat osd-config/0/whoami
run_osd: 51: '[' 0 = 0 ']'
run_osd: 53: ./ceph osd crush create-or-move 0 1 root=default host=localhost
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
create-or-move updating item name 'osd.0' weight 1 at location {host=localhost,root=default} to crush map
TEST_config_track: 57: grep 'is not > osd_map_max_advance' osd-config/osd-0.log
TEST_config_track: 58: local advance=1000
TEST_config_track: 59: local cache=500
TEST_config_track: 60: ./ceph tell osd.0 injectargs '--osd-map-cache-size 500'
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
Error ENXIO: problem getting command descriptions from osd.0
TEST_config_track: 60: return 1