Project

General

Profile

Actions

Bug #51111

closed

Pacific: CEPHADM_STRAY_DAEMON after deploying iSCSI gateway with cephadm due to tcmu-runner

Added by Paul Giralt almost 3 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
cephadm
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Deploy iscsi gateways with the command 'ceph orch apply iscsi.yaml' with the following YAML file (hostname / IP / password have been changed for privacy reasons),

service_type: iscsi
service_id: iscsi
placement:
  hosts:
    - host1.domain.com
    - host2.domain.com
    - host3.domain.com
    - host4.domain.com
spec:
  pool: iscsi-config
  trusted_ip_list: "10.10.10.10,10.10.10.11,10.10.10.12,10.10.10.10.13,10.10.10.14" 
  api_user: admin
  api_password: password_removed
  api_secure: false

All services will work properly, however the cluster health will indicate a warning whenever an image is added to a target - one for each image on each gateway (so total number of warnings will be number of gateways x number of images).

# ceph health detail
HEALTH_WARN 16 stray daemon(s) not managed by cephadm
[WRN] CEPHADM_STRAY_DAEMON: 16 stray daemon(s) not managed by cephadm
    stray daemon tcmu-runner.host2.domain.com:iscsi-pool-0001/iscsi-p0001-img-01 on host host2.domain.com not managed by cephadm
    stray daemon tcmu-runner.host2.domain.com:iscsi-pool-0002/iscsi-p0002-img-01 on host host2.domain.com not managed by cephadm
    stray daemon tcmu-runner.host2.domain.com:iscsi-pool-0003/iscsi-p0003-img-01 on host host2.domain.com not managed by cephadm
    stray daemon tcmu-runner.host2.domain.com:iscsi-pool-0004/iscsi-p0004-img-01 on host host2.domain.com not managed by cephadm
    stray daemon tcmu-runner.host3.domain.com:iscsi-pool-0001/iscsi-p0001-img-01 on host host3.domain.com not managed by cephadm
    stray daemon tcmu-runner.host3.domain.com:iscsi-pool-0002/iscsi-p0002-img-01 on host host3.domain.com not managed by cephadm
    stray daemon tcmu-runner.host3.domain.com:iscsi-pool-0003/iscsi-p0003-img-01 on host host3.domain.com not managed by cephadm
    stray daemon tcmu-runner.host3.domain.com:iscsi-pool-0004/iscsi-p0004-img-01 on host host3.domain.com not managed by cephadm
    stray daemon tcmu-runner.host4.domain.com:iscsi-pool-0001/iscsi-p0001-img-01 on host host4.domain.com not managed by cephadm
    stray daemon tcmu-runner.host4.domain.com:iscsi-pool-0002/iscsi-p0002-img-01 on host host4.domain.com not managed by cephadm
    stray daemon tcmu-runner.host4.domain.com:iscsi-pool-0003/iscsi-p0003-img-01 on host host4.domain.com not managed by cephadm
    stray daemon tcmu-runner.host4.domain.com:iscsi-pool-0004/iscsi-p0004-img-01 on host host4.domain.com not managed by cephadm
    stray daemon tcmu-runner.host5.domain.com:iscsi-pool-0001/iscsi-p0001-img-01 on host host5.domain.com not managed by cephadm
    stray daemon tcmu-runner.host5.domain.com:iscsi-pool-0002/iscsi-p0002-img-01 on host host5.domain.com not managed by cephadm
    stray daemon tcmu-runner.host5.domain.com:iscsi-pool-0003/iscsi-p0003-img-01 on host host5.domain.com not managed by cephadm
    stray daemon tcmu-runner.host5.domain.com:iscsi-pool-0004/iscsi-p0004-img-01 on host host5.domain.com not managed by cephadm

I've reproduced this twice on fresh Pacific 16.2.4 installations and I suspect it might have something to do with using FQDN.

Actions #1

Updated by Loïc Dachary almost 3 years ago

  • Target version deleted (v16.2.5)
Actions #2

Updated by Sebastian Wagner almost 3 years ago

  • Description updated (diff)
  • Category changed from cephadm/monitoring to cephadm
Actions #3

Updated by Fabian Goebel almost 3 years ago

Hello,
I have the very same Issue on a fresh install of Pacific 16.2.4 on Ubuntu with podman, but I only have used short dns names

root@ceph00:~# ceph health detail
HEALTH_WARN 2 stray daemon(s) not managed by cephadm
[WRN] CEPHADM_STRAY_DAEMON: 2 stray daemon(s) not managed by cephadm
    stray daemon tcmu-runner.ceph01:rbd/isci-test on host ceph01 not managed by cephadm
    stray daemon tcmu-runner.ceph02:rbd/isci-test on host ceph02 not managed by cephadm

Actions #4

Updated by Tobias Fischer over 2 years ago

same here as well:

HEALTH_WARN 2 stray daemon(s) not managed by cephadm
[WRN] CEPHADM_STRAY_DAEMON: 2 stray daemon(s) not managed by cephadm
    stray daemon tcmu-runner.be-iscsi20p:iscsi/test1 on host be-iscsi20p not managed by cephadm
    stray daemon tcmu-runner.be-iscsi21p:iscsi/test1 on host be-iscsi21p not managed by cephadm

Actions #5

Updated by Tobias Fischer over 2 years ago

ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable)

Actions #6

Updated by icy chan over 2 years ago

Same issue on 16.2.6 with Ubuntu 20.04.
Temporarily silence the WRAN by configuring mgr/cephadm/warn_on_stray_daemons to false.

Actions #7

Updated by Kamil Kuramshin over 2 years ago

icy chan wrote:

Same issue on 16.2.6 with Ubuntu 20.04.
Temporarily silence the WRAN by configuring mgr/cephadm/warn_on_stray_daemons to false.

Can confirm same issue. Fresh installation in Debian 11. ISCSI-Gateways deployed with ceph orch apply -i iscsi.yaml
Error raised after attempt to create new export of rdb-image,
Removing iscsi-service fix this problem.

ceph health detail
HEALTH_WARN 1 stray daemon(s) not managed by cephadm
[WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm
    stray daemon tcmu-runner.cn01:compressed_replicated/iscsi on host cn01 not managed by cephadm

Actions #8

Updated by Sebastian Wagner over 2 years ago

  • Priority changed from Normal to High
Actions #9

Updated by Sebastian Wagner over 2 years ago

  • Status changed from New to In Progress
  • Assignee set to Melissa Li
Actions #10

Updated by Melissa Li over 2 years ago

  • Pull request ID set to 43833
Actions #11

Updated by Sebastian Wagner over 2 years ago

  • Status changed from In Progress to Fix Under Review
Actions #12

Updated by Sebastian Wagner about 2 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #13

Updated by Adam King about 2 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF