Actions
Bug #16660
closed"[ERROR ] Failed to connect to host:vpm*" in ceph-deploy-hammer-distro-basic-vps
Added by Yuri Weinstein almost 8 years ago. Updated 12 months ago.
% Done:
0%
Source:
Q/A
Tags:
2016-07-12T08:39:09.847 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.gatherkeys][ERROR ] Failed to connect to host:vpm139.front.sepia.ceph.com
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-deploy
Crash signature (v1):
Crash signature (v2):
Description
For centos jobs in run: http://pulpito.ceph.com/yuriw-2016-07-12_08:00:55-ceph-deploy-hammer-distro-basic-vps/
Maybe a root cause of #11538
Logs: http://qa-proxy.ceph.com/teuthology/yuriw-2016-07-12_08:00:55-/309500/teuthology.log
2016-07-12T08:39:09.847 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.gatherkeys][ERROR ] Failed to connect to host:vpm139.front.sepia.ceph.com
Updated by Yuri Weinstein almost 8 years ago
- Description updated (diff)
- Tags set to 2016-07-12T08:39:09.847 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.gatherkeys][ERROR ] Failed to connect to host:vpm139.front.sepia.ceph.com
Updated by Zack Cerza almost 8 years ago
- Project changed from Infrastructure to 30
Updated by Zack Cerza almost 8 years ago
- Project changed from 30 to Infrastructure
Updated by David Galloway almost 8 years ago
This is the first error I see in the log.
2016-07-12T08:37:02.448 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:02.537 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:02.537 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][WARNING] monitor: mon.vpm139, might not be running yet
2016-07-12T08:37:02.540 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:02.686 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:02.687 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][WARNING] monitor vpm139 does not exist in monmap
2016-07-12T08:37:02.687 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][WARNING] neither `public_addr` nor `public_network` keys are defined for monitors
2016-07-12T08:37:02.688 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][WARNING] monitors may not be able to form quorum
2016-07-12T08:37:02.688 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][DEBUG ] detecting platform for host vpm083 ...
2016-07-12T08:37:02.784 INFO:teuthology.orchestra.run.vpm139.stderr:Warning: Permanently added 'vpm083,172.21.2.83' (ECDSA) to the list of known hosts.
2016-07-12T08:37:03.080 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] connection detected need for sudo
2016-07-12T08:37:03.152 INFO:teuthology.orchestra.run.vpm139.stderr:Warning: Permanently added 'vpm083,172.21.2.83' (ECDSA) to the list of known hosts.
2016-07-12T08:37:03.465 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] connected to host: vpm083
2016-07-12T08:37:03.466 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] detect platform information from remote host
2016-07-12T08:37:03.498 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] detect machine type
2016-07-12T08:37:03.507 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] find the location of an executable
2016-07-12T08:37:03.509 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][INFO ] distro info: CentOS Linux 7.2.1511 Core
2016-07-12T08:37:03.509 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] determining if provided host has same hostname in remote
2016-07-12T08:37:03.509 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] get remote short hostname
2016-07-12T08:37:03.511 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] deploying mon to vpm083
2016-07-12T08:37:03.512 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] get remote short hostname
2016-07-12T08:37:03.513 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] remote hostname: vpm083
2016-07-12T08:37:03.518 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
2016-07-12T08:37:03.521 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] create the mon path if it does not exist
2016-07-12T08:37:03.523 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-vpm083/done
2016-07-12T08:37:03.525 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-vpm083/done
2016-07-12T08:37:03.527 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] creating keyring file: /var/lib/ceph/tmp/ceph-vpm083.mon.keyring
2016-07-12T08:37:03.527 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] create the monitor keyring file
2016-07-12T08:37:03.543 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] Running command: sudo ceph-mon --cluster ceph --mkfs -i vpm083 --keyring /var/lib/ceph/tmp/ceph-vpm083.mon.keyring --setuser 167 --setgroup 167
2016-07-12T08:37:03.660 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] ceph-mon: mon.noname-b 172.21.2.83:6789/0 is local, renaming to mon.vpm083
2016-07-12T08:37:03.660 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] ceph-mon: set fsid to 53c9ac55-aeeb-42fb-bf53-dd09ff113379
2016-07-12T08:37:03.670 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] ceph-mon: created monfs at /var/lib/ceph/mon/ceph-vpm083 for mon.vpm083
2016-07-12T08:37:03.671 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] unlinking keyring file /var/lib/ceph/tmp/ceph-vpm083.mon.keyring
2016-07-12T08:37:03.672 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] create a done file to avoid re-doing the mon deployment
2016-07-12T08:37:03.672 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][DEBUG ] create the init path if it does not exist
2016-07-12T08:37:03.679 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] Running command: sudo systemctl enable ceph.target
2016-07-12T08:37:03.865 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] Running command: sudo systemctl enable ceph-mon@vpm083
2016-07-12T08:37:03.895 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][WARNING] Created symlink from /etc/systemd/system/ceph-mon.target.wants/ceph-mon@vpm083.service to /usr/lib/systemd/system/ceph-mon@.service.
2016-07-12T08:37:04.020 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] Running command: sudo systemctl start ceph-mon@vpm083
2016-07-12T08:37:06.162 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm083.asok mon_status
2016-07-12T08:37:06.377 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:06.377 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][WARNING] monitor: mon.vpm083, might not be running yet
2016-07-12T08:37:06.384 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm083.asok mon_status
2016-07-12T08:37:06.550 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:06.551 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][WARNING] monitor vpm083 does not exist in monmap
2016-07-12T08:37:06.551 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][WARNING] neither `public_addr` nor `public_network` keys are defined for monitors
2016-07-12T08:37:06.552 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm083][WARNING] monitors may not be able to form quorum
2016-07-12T08:37:06.552 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][INFO ] processing monitor mon.vpm139
2016-07-12T08:37:06.583 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][DEBUG ] connection detected need for sudo
2016-07-12T08:37:06.614 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][DEBUG ] connected to host: vpm139
2016-07-12T08:37:06.615 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][DEBUG ] detect platform information from remote host
2016-07-12T08:37:06.639 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][DEBUG ] detect machine type
2016-07-12T08:37:06.643 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][DEBUG ] find the location of an executable
2016-07-12T08:37:06.647 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:06.791 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:06.792 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] mon.vpm139 monitor is not yet in quorum, tries left: 5
2016-07-12T08:37:06.793 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] waiting 5 seconds before retrying
2016-07-12T08:37:11.803 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:11.893 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:11.893 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] mon.vpm139 monitor is not yet in quorum, tries left: 4
2016-07-12T08:37:11.894 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] waiting 10 seconds before retrying
2016-07-12T08:37:21.908 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:21.998 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:21.998 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] mon.vpm139 monitor is not yet in quorum, tries left: 3
2016-07-12T08:37:21.999 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] waiting 10 seconds before retrying
2016-07-12T08:37:32.013 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:32.104 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:32.106 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] mon.vpm139 monitor is not yet in quorum, tries left: 2
2016-07-12T08:37:32.106 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] waiting 15 seconds before retrying
2016-07-12T08:37:47.125 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.vpm139.asok mon_status
2016-07-12T08:37:47.214 INFO:teuthology.orchestra.run.vpm139.stderr:[vpm139][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
2016-07-12T08:37:47.214 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] mon.vpm139 monitor is not yet in quorum, tries left: 1
2016-07-12T08:37:47.214 INFO:teuthology.orchestra.run.vpm139.stderr:[ceph_deploy.mon][WARNING] waiting 20 seconds before retrying
Actions