Bug #36550: Observed time out when start Ceph development cluster enabling SPDK - Ceph - Ceph

Actions

Copy link

Bug #36550

closed

Observed time out when start Ceph development cluster enabling SPDK

Added by Anonymous over 5 years ago. Updated over 5 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

When start up the Ceph development cluster enabling SPDK with the version mater/LATEST, observed time out when start ceph-osd daemon.

The log is below:
[ DPDK EAL parameters: nvme-device-manager -c 0x1 -m 2048 --file-prefix=spdk_pid26162 ]
EAL: Detected 46 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/spdk_pid26162/mp_socket
EAL: Probing VFIO support...
EAL: VFIO support initialized
Unable to unlink shared memory file: /var/run/.spdk_pid26162_hugepage_info. Error code: 2
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 8086:953 spdk_nvme
EAL: using IOMMU type 1 (Type 1)
adding osd0 key to auth repository
/home/ubuntu/ceph/build/bin/ceph -c /home/ubuntu/ceph/build/ceph.conf -k /home/ubuntu/ceph/build/keyring -i /home/ubuntu/ceph/build/dev/osd0/keyring auth add osd.0 osd 'allow ' mon 'allow profile osd' mgr 'allow profile osd'
start osd.0
/home/ubuntu/ceph/build/bin/ceph-osd -i 0 -c /home/ubuntu/ceph/build/ceph.conf
2018-10-22 09:59:42.190 ffffb900adf0 -1 Falling back to public interface
Starting SPDK v18.04.1 / DPDK 18.05.0 initialization...
[ DPDK EAL parameters: nvme-device-manager -c 0x1 -m 2048 --file-prefix=spdk_pid26289 ]
EAL: Detected 46 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/spdk_pid26289/mp_socket
EAL: Probing VFIO support...
EAL: VFIO support initialized
Unable to unlink shared memory file: /var/run/.spdk_pid26289_hugepage_info. Error code: 2
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 8086:953 spdk_nvme
EAL: using IOMMU type 1 (Type 1)
nvme_ctrlr.c:1656:nvme_ctrlr_process_init: *ERROR: Initialization timed out in state 4
nvme_ctrlr.c: 478:nvme_ctrlr_fail: ERROR: ctrlr 0000:01:00.0 in failed state.
nvme.c: 390:nvme_init_controllers: ERROR: Failed to initialize SSD: 0000:01:00.0
EAL: Releasing pci mapped resource for 0000:01:00.0
EAL: Calling pci_unmap_resource for 0000:01:00.0 at 0xffffa0000000
2018-10-22 10:00:05.388 ffffb40fd340 -1 bdev operator() device probe nvme failed
2018-10-22 10:00:05.388 ffffb900adf0 -1 bdev() open failed to get nvme device with transport address 0000:01:00.0