Bug #41740
openmgr/dashboard: After upgrade, dashboard still shows old version sometimes
0%
Description
I recently upgraded my cluster from 14.2.2 to 14.2.3, starting with my three monitor hosts. After the third monitor host had been upgraded, I noticed the /#/hosts page still showed it as running 14.2.2, as shown in the attached screenshot. Each mon host was upgraded using "apt update ; apt upgrade" and rebooted immediately after completion of the upgrade command.
root@ceph-mon0:~# ceph versions
{
"mon": {
"ceph version 14.2.3 (4e36ac4558b09ab0ace33a4d000e83fc802f91f5) nautilus (stable)": 3
},
"mgr": {
"ceph version 14.2.3 (4e36ac4558b09ab0ace33a4d000e83fc802f91f5) nautilus (stable)": 2
},
"osd": {
"ceph version 14.2.3 (4e36ac4558b09ab0ace33a4d000e83fc802f91f5) nautilus (stable)": 16
},
"mds": {
"ceph version 14.2.3 (4e36ac4558b09ab0ace33a4d000e83fc802f91f5) nautilus (stable)": 3
},
"overall": {
"ceph version 14.2.3 (4e36ac4558b09ab0ace33a4d000e83fc802f91f5) nautilus (stable)": 24
}
}
Files
Updated by Lenz Grimmer over 4 years ago
- Subject changed from After upgrade, dashboard still shows old version sometimes to mgr/dashboard: After upgrade, dashboard still shows old version sometimes
- Description updated (diff)
Updated by Lenz Grimmer over 4 years ago
That "Version" information on the page is not very helpful to begin with. If multiple services are running on a single host, their versions might actually be different from each other - this needs to be improved.
Would it be possible for you to send us the output of the following two API GET requests?
https://<dashboard host IP>/api/host https://<dashboard host IP>/api/summary
You can perform these API requests by clicking on the help icon (question mark) in the top right corner and selecting "API" from the dropdown.
Updated by Lenz Grimmer over 4 years ago
- Related to Bug #41790: mgr/dashboard: Only one version number per host displayed added
Updated by Lenz Grimmer over 4 years ago
FWIW, I now submitted #41790 to request a change in how the version information is displayed.
Updated by Sebastian Wagner over 4 years ago
- Related to Bug #41869: mgr/dashboard: api/mgr/module resource is returning an internal server error added
Updated by Sebastian Wagner over 4 years ago
- Related to deleted (Bug #41869: mgr/dashboard: api/mgr/module resource is returning an internal server error)
Updated by Sebastian Wagner over 4 years ago
- Related to Bug #41878: mgr/prometheus: KeyError in mgr/prometheus/module.py", line 490, in get_mgr_status added
Updated by Sebastian Wagner over 4 years ago
mgr_module.MgrModule#list_servers might return outdated information. This would be a good explanation for a similar issue #41878
Updated by Charley Sheets over 4 years ago
I haven't been able to provide the requested information because the problem went away after a few mgr reboots. However, now that I'll be upgrading to 14.2.4, I will see if I can reproduce the problem and then provide the requested information.
Updated by Lenz Grimmer over 4 years ago
- Status changed from New to Need More Info
Charley Sheets wrote:
I haven't been able to provide the requested information because the problem went away after a few mgr reboots. However, now that I'll be upgrading to 14.2.4, I will see if I can reproduce the problem and then provide the requested information.
Thanks! Setting this to "Need more info" for the time being.
Updated by Charley Sheets about 4 years ago
After upgrading one of my three mon hosts from 14.2.5-1-g23e76c7aa6-1 to 14.2.6-1-gffd69200ad-1, it is still reporting its version as 14.2.5 on both the web and via the API. I've attached a screenshot. I'm using the Debian packages published by croit.io.
Here are the API call results that were previously requested:
Request URL: https://ceph.picote.ch/api/host
Response:
[
{
"services": [
{
"type": "mds",
"id": "ceph-mon0"
},
{
"type": "mgr",
"id": "ceph-mon0"
},
{
"type": "mon",
"id": "ceph-mon0"
}
],
"hostname": "ceph-mon0",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
},
{
"services": [
{
"type": "mds",
"id": "ceph-mon1"
},
{
"type": "mon",
"id": "ceph-mon1"
}
],
"hostname": "ceph-mon1",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
},
{
"services": [
{
"type": "mds",
"id": "ceph-mon2"
},
{
"type": "mgr",
"id": "ceph-mon2"
},
{
"type": "mon",
"id": "ceph-mon2"
}
],
"hostname": "ceph-mon2",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
},
{
"services": [
{
"type": "osd",
"id": "2"
},
{
"type": "osd",
"id": "3"
}
],
"hostname": "ceph-node1",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
},
{
"services": [
{
"type": "osd",
"id": "4"
},
{
"type": "osd",
"id": "5"
}
],
"hostname": "ceph-node101",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
},
{
"services": [
{
"type": "osd",
"id": "10"
},
{
"type": "osd",
"id": "11"
},
{
"type": "osd",
"id": "12"
},
{
"type": "osd",
"id": "6"
},
{
"type": "osd",
"id": "7"
},
{
"type": "osd",
"id": "8"
},
{
"type": "osd",
"id": "9"
}
],
"hostname": "ceph-node2",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
},
{
"services": [
{
"type": "osd",
"id": "13"
},
{
"type": "osd",
"id": "15"
},
{
"type": "osd",
"id": "16"
}
],
"hostname": "ceph-node3",
"ceph_version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)"
}
]
Request URL: https://ceph.picote.ch/api/summary
Response:
{
"rbd_mirroring": {
"errors": 0,
"warnings": 0
},
"finished_tasks": [],
"executing_tasks": [],
"health_status": "HEALTH_WARN",
"version": "ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)",
"mgr_id": "ceph-mon2",
"have_mon_connection": true,
"mgr_host": "https://ceph-mon2.picote.ch:8443/"
}
Finally, ceph-mon0 (the host which was upgraded and then rebooted before collecting the above information) reports the following version information from the CLI:
root@ceph-mon0:~# /usr/bin/ceph-mgr --version ceph version 14.2.6-1-gffd69200ad (ffd69200ad0fc59d919bacc5a0f2a09f24be4134) nautilus (stable)
Updated by Charley Sheets about 4 years ago
The versions reported by the ceph versions command are what I expect. This is after upgrading the second of three mon servers:
{
"mon": {
"ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)": 1,
"ceph version 14.2.6-1-gffd69200ad (ffd69200ad0fc59d919bacc5a0f2a09f24be4134) nautilus (stable)": 2
},
"mgr": {
"ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)": 1,
"ceph version 14.2.6-1-gffd69200ad (ffd69200ad0fc59d919bacc5a0f2a09f24be4134) nautilus (stable)": 1
},
"osd": {
"ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)": 12
},
"mds": {
"ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)": 1,
"ceph version 14.2.6-1-gffd69200ad (ffd69200ad0fc59d919bacc5a0f2a09f24be4134) nautilus (stable)": 1
},
"overall": {
"ceph version 14.2.5-1-g23e76c7aa6 (23e76c7aa6e15817ffb6741aafbc95ca99f24cbb) nautilus (stable)": 15,
"ceph version 14.2.6-1-gffd69200ad (ffd69200ad0fc59d919bacc5a0f2a09f24be4134) nautilus (stable)": 4
}
}
Updated by Ernesto Puerta about 3 years ago
- Project changed from mgr to Dashboard
- Category changed from 132 to General