Bug #46910
cephadm: Unable to create an iSCSI target
0%
Description
When I try to create an iSCSI target I get the following error:
On `ceph-iscsi` container logs I see:
Aug 11 16:27:02 node1 bash[26919]: debug ::1 - - [11/Aug/2020 14:27:02] "GET /api/_ping HTTP/1.1" 200 - Aug 11 16:27:03 node1 bash[26919]: debug (LUN.add_dev_to_lio) Adding image 'mypool/myrbd1' to LIO backstore user:rbd Aug 11 16:27:03 node1 bash[26919]: debug failed to add mypool/myrbd1 to LIO - error([Errno 2] No such file or directory) Aug 11 16:27:03 node1 bash[26919]: debug LUN alloc problem - failed to add mypool/myrbd1 to LIO - error([Errno 2] No such file or directory)
My service spec:
# tail -14 service_spec_gw.yml --- service_type: iscsi service_id: iscsi_service placement: hosts: - 'node1' - 'node2' spec: pool: rbd trusted_ip_list: 10.20.139.201,10.20.139.202,10.20.139.203 api_port: 5000 api_user: admin1 api_password: admin2 api_secure: False
My test environment was created using the following `sesdev` command:
sesdev create pacific --ceph-salt-repo https://github.com/ceph/ceph-salt.git --ceph-salt-branch master pacific
History
#1 Updated by Matthew Oliver over 3 years ago
I think I've had this before, if I remember correctly I've seen this when the tcmu-runner container was running but didn't have /dev volume mounted into the container.
What does your tcmu-runner unit.run look like?
#2 Updated by Ricardo Marques over 3 years ago
node1:~ # podman ps | grep tcmu aad4b80fe431 registry.opensuse.org/filesystems/ceph/master/upstream/images/ceph/ceph:latest /bin/bash --login 3 minutes ago Up 3 minutes ago ceph-28341724-dc8b-11ea-bb36-5254009df885-iscsi.iscsi_service.node1.kghzlz-tcmu node1:~ # podman exec -ti aad4b80fe431 /bin/bash node1:/ # ps -ef | grep tcmu root 1 0 0 13:02 ? 00:00:00 /usr/bin/tcmu-runner root 48 10 0 13:09 pts/1 00:00:00 grep tcmu
node1:/ # cat /var/log/tcmu-runner.log 2020-08-13 13:02:31.499 1 [INFO] load_our_module:548: Inserted module 'target_core_user' 2020-08-13 13:04:35.675 1 [ERROR] add_device:456: could not find handler for uio0 2020-08-13 13:04:35.677 1 [ERROR] add_device:456: could not find handler for uio0
node1:/ # ls /dev autofs dm-1 lightnvm port stdin tty18 tty3 tty41 tty53 tty8 ttyS19 ttyS30 vcs vcsa6 vdc block dri log ppp stdout tty19 tty30 tty42 tty54 tty9 ttyS2 ttyS31 vcs1 vcsu vfio btrfs-control fb0 loop-control psaux tty tty2 tty31 tty43 tty55 ttyS0 ttyS20 ttyS4 vcs2 vcsu1 vga_arbiter bus fd mapper ptmx tty0 tty20 tty32 tty44 tty56 ttyS1 ttyS21 ttyS5 vcs3 vcsu2 vhci ceph-19b86412-4dec-4956-bdef-bde1259f60a3 full mcelog pts tty1 tty21 tty33 tty45 tty57 ttyS10 ttyS22 ttyS6 vcs4 vcsu3 vhost-net ceph-4d581afd-a93b-4553-95aa-4d31904167b3 fuse mem random tty10 tty22 tty34 tty46 tty58 ttyS11 ttyS23 ttyS7 vcs5 vcsu4 vhost-vsock char hpet memory_bandwidth rfkill tty11 tty23 tty35 tty47 tty59 ttyS12 ttyS24 ttyS8 vcs6 vcsu5 zero console hugepages mqueue rtc tty12 tty24 tty36 tty48 tty6 ttyS13 ttyS25 ttyS9 vcsa vcsu6 core hwrng net rtc0 tty13 tty25 tty37 tty49 tty60 ttyS14 ttyS26 udmabuf vcsa1 vda cpu_dma_latency initctl network_latency shm tty14 tty26 tty38 tty5 tty61 ttyS15 ttyS27 uhid vcsa2 vda1 cuse input network_throughput snapshot tty15 tty27 tty39 tty50 tty62 ttyS16 ttyS28 uinput vcsa3 vda2 disk kmsg null snd tty16 tty28 tty4 tty51 tty63 ttyS17 ttyS29 urandom vcsa4 vda3 dm-0 kvm nvram stderr tty17 tty29 tty40 tty52 tty7 ttyS18 ttyS3 userio vcsa5 vdb node1:/ # exit node1:~ # ls /dev autofs dm-1 lightnvm port stdin tty18 tty3 tty41 tty53 tty8 ttyS19 ttyS30 vcs vcsa6 vdc block dri log ppp stdout tty19 tty30 tty42 tty54 tty9 ttyS2 ttyS31 vcs1 vcsu vfio btrfs-control fb0 loop-control psaux tty tty2 tty31 tty43 tty55 ttyS0 ttyS20 ttyS4 vcs2 vcsu1 vga_arbiter bus fd mapper ptmx tty0 tty20 tty32 tty44 tty56 ttyS1 ttyS21 ttyS5 vcs3 vcsu2 vhci ceph-19b86412-4dec-4956-bdef-bde1259f60a3 full mcelog pts tty1 tty21 tty33 tty45 tty57 ttyS10 ttyS22 ttyS6 vcs4 vcsu3 vhost-net ceph-4d581afd-a93b-4553-95aa-4d31904167b3 fuse mem random tty10 tty22 tty34 tty46 tty58 ttyS11 ttyS23 ttyS7 vcs5 vcsu4 vhost-vsock char hpet memory_bandwidth rfkill tty11 tty23 tty35 tty47 tty59 ttyS12 ttyS24 ttyS8 vcs6 vcsu5 zero console hugepages mqueue rtc tty12 tty24 tty36 tty48 tty6 ttyS13 ttyS25 ttyS9 vcsa vcsu6 core hwrng net rtc0 tty13 tty25 tty37 tty49 tty60 ttyS14 ttyS26 udmabuf vcsa1 vda cpu_dma_latency initctl network_latency shm tty14 tty26 tty38 tty5 tty61 ttyS15 ttyS27 uhid vcsa2 vda1 cuse input network_throughput snapshot tty15 tty27 tty39 tty50 tty62 ttyS16 ttyS28 uinput vcsa3 vda2 disk kmsg null snd tty16 tty28 tty4 tty51 tty63 ttyS17 ttyS29 urandom vcsa4 vda3 dm-0 kvm nvram stderr tty17 tty29 tty40 tty52 tty7 ttyS18 ttyS3 userio vcsa5 vdb
#3 Updated by Sebastian Wagner over 3 years ago
- Assignee set to Matthew Oliver
#4 Updated by Matthew Oliver over 3 years ago
- Assignee deleted (
Matthew Oliver)
Finally tracked down the issue. Turns out our suse ceph image doesn't contain the tcmu-runner-handler-rbd package. If I install that in the container it works.
This is the handler that allows tcmu-runner to talk to librbd.
Next I'll need to go figure out where it's built, I assume obs/ibs.
#5 Updated by Matthew Oliver over 3 years ago
- Assignee set to Matthew Oliver
#6 Updated by Matthew Oliver over 3 years ago
Created an open build system (OBS) request (PR) to add the missing pacakage to the ceph-base pattern used in building the Suse container: https://build.opensuse.org/request/show/830139
#7 Updated by Matthew Oliver over 3 years ago
The first OBS request has been accepted. The suse product container doesn't suffer from the same problem. It seems it's only the OBS/LEAP container side.
#8 Updated by Sebastian Wagner over 3 years ago
- Status changed from New to Can't reproduce