Bug #51446
closedModule 'dashboard' has failed: Timeout('Port 8443 not bound on 192.168.122.121.',)
0%
Description
standby dashboard is binding to 127.0.1.1 becasue get_mgr_ip() always returns hostname for standby mgr and with podmans /etc/host file the hostname resolves to 127.0.1.1 when failing over to the standby mgr it crashes cause cherry py is still binded to 127.0.1.1 when it expects it to be binded to the mgr ip
Updated by Sage Weil almost 3 years ago
- Project changed from Ceph to Orchestrator
Updated by Patrick Seidensal almost 3 years ago
This might not be a pure orchestrator issue. I've investigated the `run-backend-api-tests.sh` issue I have locally and the cause seems to be the same. As the change to determine the host name or IP address of the dashboard module is about a month old, I suspect that the upstream tests have not suffered by that change, but it might be the cause.
https://github.com/ceph/ceph/commit/157a7b4183dbd888f106c613a758409d7e07b917
Whatever might fix that incident for podman, it seems it'll do so for running the Teuthology tests locally, too.
2021-06-30T16:52:11.364+0200 7f3da7c08700 -1 Traceback (most recent call last): File "/home/user/src/ceph/src/pybind/mgr/dashboard/module.py", line 343, in serve cherrypy.engine.start() File "/usr/lib/python3/dist-packages/cherrypy/process/wspbus.py", line 264, in start self.exit() File "/usr/lib/python3/dist-packages/cherrypy/process/wspbus.py", line 254, in start self.publish('start') File "/usr/lib/python3/dist-packages/cherrypy/process/wspbus.py", line 234, in publish raise exc cherrypy.process.wspbus.ChannelFailures: OSError("Port 8890 not bound on '192.168.1.140'") LISTEN 0 5 192.168.1.140:8889 0.0.0.0:* users:(("ceph-mgr",pid=66141,fd=40)) LISTEN 0 5 127.0.1.1:8890 0.0.0.0:* users:(("ceph-mgr",pid=66129,fd=24)) LISTEN 0 5 127.0.1.1:8891 0.0.0.0:* users:(("ceph-mgr",pid=66134,fd=24))
It just works when I tell the dashboard module to listen on '0.0.0.0'.
(Note that I experimented with the ports and had them starting with 8889 instead of 7789 on my system currently)
Updated by Daniel Pivonka almost 3 years ago
- Status changed from New to In Progress
- Pull request ID set to 42107
Updated by Patrick Seidensal almost 3 years ago
- Related to Bug #37651: Container, Mimic: Unable to open CherryPy port added
Updated by Patrick Seidensal almost 3 years ago
- Related to Bug #38411: Under rook+minikube: HEALTH_ERR: Module 'dashboard' has failed: IOError("Port 8443 not bound on '::'",) added
Updated by Sage Weil almost 3 years ago
- Status changed from In Progress to Resolved
Updated by Roland Sommer almost 3 years ago
Despite being closed, the change does affect certain setups using HA-IPs and/or IPVS because contrary to the docs saying that the dashboard will listen on every (local) IP if not configured otherwise, it now does not anymore. We fixed this using '0.0.0.0' for server_addr, but the docs should be adjusted if this is the new expected behaviour, because the following is just not correct anymore:
If no specific address has been configured, the web app will bind to ::, which corresponds to all available IPv4 and IPv6 addresses.
Updated by Roland Sommer almost 3 years ago
The same for all other places, where now get_mgr_ip is used. For example the prometheus module: docs say default is 0.0.0.0 - which is also not true anymore, breaking configurations relying on the any-listen-semantics.