Project

General

Profile

Actions

Bug #50830

closed

rgw-ingress does not install

Added by Kefu Chai almost 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

rados:cephadm:smoke-roleless/{0-distro/centos_8.2_kubic_stable 1-start 2-services/rgw-ingress 3-final}

2021-05-16T15:09:57.997 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: 2021-05-16T15:09:56.708283+0000 mon.smithi106 (mon.0)
2021-05-16T15:09:57.997 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: 635 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in
2021-05-16T15:09:57.997 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: audit 2021-05-16T15:09:56
2021-05-16T15:09:57.998 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: .714931+0000 mon.smithi106 (mon.0) 636 : audit
2021-05-16T15:09:57.998 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: [INF] from='mgr.14170 172.21.15.106:0/244332724' entity='mgr.smithi106.ntuyco'
2021-05-16T15:09:57.998 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: cephadm 2021-05-16T15:09:56
2021-05-16T15:09:57.998 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: .717877+0000 mgr.smithi106.ntuyco (mgr.14170) 268 : cephadm
2021-05-16T15:09:57.998 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:57 smithi134 conmon[28912]: [INF] Deploying daemon haproxy.rgw.foo.smithi134.dzswov on smithi134
2021-05-16T15:09:58.992 INFO:journalctl@ceph.mon.smithi134.smithi134.stdout:May 16 15:09:58 smithi134 conmon[28912]: cluster 2021-05-16T15:09:57.707101+0000 mon.smithi106 (mon.0) 637 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in
2021-05-16T15:09:58.994 INFO:journalctl@ceph.mon.smithi106.smithi106.stdout:May 16 15:09:58 smithi106 conmon[27412]: cluster 2021-05-16T15:09:57.707101+0000 mon.smithi106 (mon.0) 637 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in
2021-05-16T15:09:59.583 INFO:teuthology.orchestra.run.smithi106.stdout:
2021-05-16T15:09:59.583 INFO:teuthology.orchestra.run.smithi106.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2021-05-16T15:03:46.472140Z", "last_refresh": "2021-05-16T15:07:20.577225Z", "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2021-05-16T15:03:33.706303Z", "last_refresh": "2021-05-16T15:07:20.577283Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "status": {"created": "2021-05-16T15:03:43.478569Z", "last_refresh": "2021-05-16T15:07:20.577353Z", "running": 1, "size": 1}}, {"events": ["2021-05-16T15:09:32.508962Z service:ingress.rgw.foo [INFO] \"service was created\""], "placement": {"count": 2}, "service_id": "rgw.foo", "service_name": "ingress.rgw.foo", "service_type": "ingress", "spec": {"backend_service": "rgw.foo", "frontend_port": 9000, "monitor_port": 9001, "virtual_ip": "10.0.31.106/16"}, "status": {"created": "2021-05-16T15:09:32.503551Z", "ports": [9000, 9001], "running": 1, "size": 4, "virtual_ip": "10.0.31.106/16"}}, {"placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2021-05-16T15:03:32.096682Z", "last_refresh": "2021-05-16T15:07:20.577162Z", "running": 2, "size": 2}}, {"events": ["2021-05-16T15:05:02.214320Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi106:172.21.15.106=smithi106", "smithi134:172.21.15.134=smithi134"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2021-05-16T15:05:02.211445Z", "last_refresh": "2021-05-16T15:07:20.577004Z", "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2021-05-16T15:03:45.116043Z", "last_refresh": "2021-05-16T15:07:20.577406Z", "running": 2, "size": 2}}, {"placement": {}, "service_id": "unmanaged", "service_name": "osd.unmanaged", "service_type": "osd", "spec": {"filter_logic": "AND", "objectstore": "bluestore"}, "status": {"container_image_id": "64b9af15c50d7bcae279174ce7c2c6d1c681487e2c644a679c2ff99af1278fc5", "container_image_name": "quay.ceph.io/ceph-ci/ceph@sha256:78e23af8ecdb6b1ca8d5e9eb6851e199b6397e4fbe70a69ae912ad3c61dabce5", "last_refresh": "2021-05-16T15:07:20.577512Z", "running": 8, "size": 8}, "unmanaged": true}, {"placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2021-05-16T15:03:41.761902Z", "last_refresh": "2021-05-16T15:07:20.577460Z", "running": 1, "size": 1}}, {"events": ["2021-05-16T15:09:32.503293Z service:rgw.foo [INFO] \"service was created\""], "placement": {"count": 4, "host_pattern": "*"}, "service_id": "foo", "service_name": "rgw.foo", "service_type": "rgw", "spec": {"rgw_frontend_port": 8000}, "status": {"created": "2021-05-16T15:09:34.038752Z", "ports": [8000], "running": 4, "size": 4}}]
...
2021-05-16T15:10:00.345 INFO:tasks.cephadm:ingress.rgw.foo has 1/4
...
2021-05-16T15:10:22.539 INFO:tasks.cephadm:ingress.rgw.foo has 4/4
2021-05-16T15:09:50.206 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service...
2021-05-16T15:09:50.217 INFO:tasks.cephadm:Waiting for ceph service ingress.rgw.foo to start (timeout 300)...
...
2021-05-16T15:27:58.326 INFO:teuthology.orchestra.run.smithi106.stderr:^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 10.0.31.106 port 9000: Connection refused
...2021-05-16T15:27:59.332 INFO:teuthology.orchestra.run.smithi106.stderr:^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 10.0.31.106 port 9000: Connection refused
...
2021-05-16T15:28:00.337 INFO:teuthology.orchestra.run.smithi106.stderr:^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 10.0.31.106 port 9000: Connection refused
...

/a/kchai-2021-05-16_14:47:35-rados:cephadm:smoke-roleless-master-distro-basic-smithi/6117434

Actions #1

Updated by Kefu Chai almost 3 years ago

this failure is reproducible .

Actions #2

Updated by Kefu Chai almost 3 years ago

i tried to rebuild https://github.com/ceph/ceph/commit/1a55d822295f03309a696a301bbd1314953974a1, which is the merge commit of https://github.com/ceph/ceph/pull/40888, and rerun the failed test at https://pulpito.ceph.com/kchai-2021-05-17_02:45:05-rados:cephadm:smoke-roleless-wip-pr-40888-kefu-distro-basic-smithi/ . i was stilling seeing

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 10.0.31.110 port 9000: Connection refused

i killed the job anyway.

Actions #3

Updated by Kefu Chai almost 3 years ago

  • Assignee set to Sage Weil

Sage, could you kindly take a look at this failure?

Actions #4

Updated by Sage Weil almost 3 years ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 41370
Actions #5

Updated by Sage Weil almost 3 years ago

  • Status changed from Fix Under Review to Pending Backport
  • Backport set to pacific
Actions #6

Updated by Sage Weil almost 3 years ago

  • Status changed from Pending Backport to Resolved
Actions #7

Updated by Aishwarya Mathuria over 2 years ago

/a/yuriw-2022-01-13_18:06:52-rados-wip-yuri3-testing-2022-01-13-0809-distro-default-smithi/6614359

Actions

Also available in: Atom PDF