Actions
Bug #47702
closedupgrading via ceph orch upgrade start results in partial application and mixed state
Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Following https://docs.ceph.com/en/latest/cephadm/upgrade/#using-customized-container-images I attempted to upgrade my cluster.
The process starts fine but I end up in a weird state with two mgr daemons upgraded, the upgrade seemingly succeeded and a HEALTH_WARN.
Starting with a healthy cluster at version 15.2.4-944-g85788353cf (SUSE downstream container) I run ceph orch upgrade start --image <custom registry url>/containers/ses/7/containers/ses/7/ceph/ceph:15.2.5-220-gb758bfd693
. This starts the process alright and I can see the progress of the image pull in ceph -s.
After a while this finishes and left the cluster in the following state:
master:~ # ceph -s cluster: id: 4405d2ce-031b-11eb-a7e5-525400088cac health: HEALTH_WARN 1 hosts fail cephadm check services: mon: 3 daemons, quorum master,node2,node1 (age 113m) mgr: node2.toaphn(active, since 17m), standbys: master.pthpjq, node1.tcqcfr osd: 20 osds: 20 up (since 112m), 20 in (since 112m) data: pools: 1 pools, 1 pgs objects: 0 objects, 0 B usage: 20 GiB used, 140 GiB / 160 GiB avail pgs: 1 active+clean master:~ # ceph versions { "mon": { "ceph version 15.2.4-944-g85788353cf (85788353cfa5b673d4966d4748513c33dbee228e) octopus (stable)": 3 }, "mgr": { "ceph version 15.2.4-944-g85788353cf (85788353cfa5b673d4966d4748513c33dbee228e) octopus (stable)": 1, "ceph version 15.2.5-220-gb758bfd693 (b758bfd69359a0ffa10bd5426d64e7636bb0a6c6) octopus (stable)": 2 }, "osd": { "ceph version 15.2.4-944-g85788353cf (85788353cfa5b673d4966d4748513c33dbee228e) octopus (stable)": 20 }, "mds": {}, "overall": { "ceph version 15.2.4-944-g85788353cf (85788353cfa5b673d4966d4748513c33dbee228e) octopus (stable)": 24, "ceph version 15.2.5-220-gb758bfd693 (b758bfd69359a0ffa10bd5426d64e7636bb0a6c6) octopus (stable)": 2 } }
The current active mgr could not be failed.
master:~ # ceph mgr fail toaphn Daemon not found 'toaphn', already failed?
Actions