Actions
Bug #7254
closed"pthread lock: Invalid argument" when deploying monitors
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
2014-01-28T02:38:28.846 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm099[0m][[1;37mINFO[0m ] Running command: sudo ceph --version 2014-01-28T02:38:28.966 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm099[0m][[1;34mDEBUG[0m ] ceph version 0.75-239-g9e52398 (9e523984a9e134e370153e5c47b13c49f5024e56) 2014-01-28T02:38:28.982 DEBUG:teuthology.orchestra.run:Running [10.214.138.101]: 'cd /home/ubuntu/cephtest/ceph-deploy && ./ceph-deploy mon create-initial' 2014-01-28T02:38:29.229 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mceph_deploy.cli[0m][[1;37mINFO[0m ] Invoked (1.3.4): ./ceph-deploy mon create-initial 2014-01-28T02:38:29.231 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mceph_deploy.mon[0m][[1;34mDEBUG[0m ] Deploying mon, cluster ceph hosts vpm096 vpm055 vpm098 2014-01-28T02:38:29.231 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mceph_deploy.mon[0m][[1;34mDEBUG[0m ] detecting platform for host vpm096 ... 2014-01-28T02:38:29.291 INFO:teuthology.orchestra.run.err:[10.214.138.101]: Warning: Permanently added 'vpm096,10.214.138.143' (RSA) to the list of known hosts. 2014-01-28T02:38:29.517 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] connected to host: vpm096 2014-01-28T02:38:29.518 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] detect platform information from remote host 2014-01-28T02:38:29.543 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] detect machine type 2014-01-28T02:38:29.549 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mceph_deploy.mon[0m][[1;37mINFO[0m ] distro info: Fedora 19 Schrödinger’s Cat 2014-01-28T02:38:29.549 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] determining if provided host has same hostname in remote 2014-01-28T02:38:29.549 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] get remote short hostname 2014-01-28T02:38:29.551 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] deploying mon to vpm096 2014-01-28T02:38:29.551 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] get remote short hostname 2014-01-28T02:38:29.553 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] remote hostname: vpm096 2014-01-28T02:38:29.554 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] write cluster configuration to /etc/ceph/{cluster}.conf 2014-01-28T02:38:29.557 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] create the mon path if it does not exist 2014-01-28T02:38:29.558 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] checking for done path: /var/lib/ceph/mon/ceph-vpm096/done 2014-01-28T02:38:29.559 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] done path does not exist: /var/lib/ceph/mon/ceph-vpm096/done 2014-01-28T02:38:29.561 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;37mINFO[0m ] creating keyring file: /var/lib/ceph/tmp/ceph-vpm096.mon.keyring 2014-01-28T02:38:29.561 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] create the monitor keyring file 2014-01-28T02:38:29.565 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;37mINFO[0m ] Running command: sudo ceph-mon --cluster ceph --mkfs -i vpm096 --keyring /var/lib/ceph/tmp/ceph-vpm096.mon.keyring 2014-01-28T02:38:29.740 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] pthread lock: Invalid argument 2014-01-28T02:38:29.740 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] *** Caught signal (Aborted) ** 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] in thread 7f6465b427c0 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] ceph version 0.75-239-g9e52398 (9e523984a9e134e370153e5c47b13c49f5024e56) 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 1: ceph-mon() [0x82ff9e] 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2: (()+0xefa0) [0x7f6464f44fa0] 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 3: (gsignal()+0x39) [0x7f6463b17a19] 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 4: (abort()+0x148) [0x7f6463b19128] 2014-01-28T02:38:29.741 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 5: (()+0x13eb3) [0x7f64646d1eb3] 2014-01-28T02:38:29.742 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 6: (leveldb::port::Mutex::Unlock()+0) [0x7f64646f4340] 2014-01-28T02:38:29.743 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 7: (leveldb::DBImpl::~DBImpl()+0x2a) [0x7f64646d770a] 2014-01-28T02:38:29.743 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 8: (leveldb::DBImpl::~DBImpl()+0x9) [0x7f64646d7b29] 2014-01-28T02:38:29.743 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 9: (LevelDBStore::LevelDBWholeSpaceIteratorImpl::lower_bound(std::string const&, std::string const&)+0x39) [0x801659] 2014-01-28T02:38:29.743 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 10: (MonitorDBStore::exists(std::string const&, std::string const&)+0xd9) [0x534199] 2014-01-28T02:38:29.743 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 11: (Monitor::check_fsid()+0x4d) [0x54ac7d] 2014-01-28T02:38:29.743 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 12: (Monitor::mkfs(ceph::buffer::list&)+0x34) [0x54c394] 2014-01-28T02:38:29.744 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 13: (main()+0x703) [0x523b13] 2014-01-28T02:38:29.744 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 14: (__libc_start_main()+0xf5) [0x7f6463b03b75] 2014-01-28T02:38:29.745 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 15: ceph-mon() [0x530899] 2014-01-28T02:38:29.745 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2014-01-28 10:38:29.724910 7f6465b427c0 -1 *** Caught signal (Aborted) ** 2014-01-28T02:38:29.745 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] in thread 7f6465b427c0 2014-01-28T02:38:29.745 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2014-01-28T02:38:29.747 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] ceph version 0.75-239-g9e52398 (9e523984a9e134e370153e5c47b13c49f5024e56) 2014-01-28T02:38:29.748 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 1: ceph-mon() [0x82ff9e] 2014-01-28T02:38:29.748 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2: (()+0xefa0) [0x7f6464f44fa0] 2014-01-28T02:38:29.748 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 3: (gsignal()+0x39) [0x7f6463b17a19] 2014-01-28T02:38:29.748 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 4: (abort()+0x148) [0x7f6463b19128] 2014-01-28T02:38:29.748 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 5: (()+0x13eb3) [0x7f64646d1eb3] 2014-01-28T02:38:29.748 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 6: (leveldb::port::Mutex::Unlock()+0) [0x7f64646f4340] 2014-01-28T02:38:29.749 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 7: (leveldb::DBImpl::~DBImpl()+0x2a) [0x7f64646d770a] 2014-01-28T02:38:29.751 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 8: (leveldb::DBImpl::~DBImpl()+0x9) [0x7f64646d7b29] 2014-01-28T02:38:29.751 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 9: (LevelDBStore::LevelDBWholeSpaceIteratorImpl::lower_bound(std::string const&, std::string const&)+0x39) [0x801659] 2014-01-28T02:38:29.751 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 10: (MonitorDBStore::exists(std::string const&, std::string const&)+0xd9) [0x534199] 2014-01-28T02:38:29.751 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 11: (Monitor::check_fsid()+0x4d) [0x54ac7d] 2014-01-28T02:38:29.751 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 12: (Monitor::mkfs(ceph::buffer::list&)+0x34) [0x54c394] 2014-01-28T02:38:29.751 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 13: (main()+0x703) [0x523b13] 2014-01-28T02:38:29.752 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 14: (__libc_start_main()+0xf5) [0x7f6463b03b75] 2014-01-28T02:38:29.752 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 15: ceph-mon() [0x530899] 2014-01-28T02:38:29.752 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2014-01-28T02:38:29.752 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2014-01-28T02:38:29.752 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 0> 2014-01-28 10:38:29.724910 7f6465b427c0 -1 *** Caught signal (Aborted) ** 2014-01-28T02:38:29.752 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] in thread 7f6465b427c0 2014-01-28T02:38:29.754 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2014-01-28T02:38:29.754 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] ceph version 0.75-239-g9e52398 (9e523984a9e134e370153e5c47b13c49f5024e56) 2014-01-28T02:38:29.754 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 1: ceph-mon() [0x82ff9e] 2014-01-28T02:38:29.754 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2: (()+0xefa0) [0x7f6464f44fa0] 2014-01-28T02:38:29.754 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 3: (gsignal()+0x39) [0x7f6463b17a19] 2014-01-28T02:38:29.755 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 4: (abort()+0x148) [0x7f6463b19128] 2014-01-28T02:38:29.755 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 5: (()+0x13eb3) [0x7f64646d1eb3] 2014-01-28T02:38:29.755 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 6: (leveldb::port::Mutex::Unlock()+0) [0x7f64646f4340] 2014-01-28T02:38:29.755 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 7: (leveldb::DBImpl::~DBImpl()+0x2a) [0x7f64646d770a] 2014-01-28T02:38:29.755 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 8: (leveldb::DBImpl::~DBImpl()+0x9) [0x7f64646d7b29] 2014-01-28T02:38:29.755 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 9: (LevelDBStore::LevelDBWholeSpaceIteratorImpl::lower_bound(std::string const&, std::string const&)+0x39) [0x801659] 2014-01-28T02:38:29.757 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 10: (MonitorDBStore::exists(std::string const&, std::string const&)+0xd9) [0x534199] 2014-01-28T02:38:29.757 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 11: (Monitor::check_fsid()+0x4d) [0x54ac7d] 2014-01-28T02:38:29.757 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 12: (Monitor::mkfs(ceph::buffer::list&)+0x34) [0x54c394] 2014-01-28T02:38:29.757 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 13: (main()+0x703) [0x523b13] 2014-01-28T02:38:29.757 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 14: (__libc_start_main()+0xf5) [0x7f6463b03b75] 2014-01-28T02:38:29.758 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 15: ceph-mon() [0x530899] 2014-01-28T02:38:29.758 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2014-01-28T02:38:29.758 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;33mWARNIN[0m] 2014-01-28T02:38:29.758 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] ceph-mon: mon.noname-a 10.214.138.143:6789/0 is local, renaming to mon.vpm096 2014-01-28T02:38:29.758 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;34mDEBUG[0m ] ceph-mon: set fsid to 8492734e-1530-4eea-9680-039d94556a64 2014-01-28T02:38:29.759 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mvpm096[0m][[1;31mERROR[0m ] RuntimeError: command returned non-zero exit status: -6 2014-01-28T02:38:29.759 INFO:teuthology.orchestra.run.err:[10.214.138.101]: [[1mceph_deploy.mon[0m][[1;31mERROR[0m ] Failed to execute command: ceph-mon --cluster ceph --mkfs -i vpm096 --keyring /var/lib/ceph/tmp/ceph-vpm096.mon.keyring
Actions