Actions
Bug #53923
closed[Upgrade] mgr FAILED to decode MSG_PGSTATS
% Done:
100%
Source:
Tags:
Backport:
quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
# ceph -s cluster: id: 182eef00-53b5-11ec-84d3-3cecef3d8fb8 health: HEALTH_WARN Reduced data availability: 65553 pgs inactive 1 mgr modules have recently crashed services: mon: 5 daemons, quorum gibba001,gibba002,gibba004,gibba005,gibba006 (age 30m) mgr: gibba001.zptzqf(active, since 39m), standbys: gibba002.veobjs mds: 1/1 daemons up, 1 standby osd: 975 osds: 975 up (since 86m), 975 in (since 54m) data: volumes: 1/1 healthy pools: 4 pools, 65553 pgs objects: 0 objects, 0 B usage: 0 B used, 0 B / 0 B avail pgs: 100.000% pgs unknown 65553 unknown progress: Global Recovery Event (0s) [............................] Upgrade to 17.0.0-10229-g7e035110 (6m) [............................] (remaining: 3h)
- MGR logs:
2022-01-18T19:54:52.484+0000 7f8914f04000 0 ceph version 17.0.0-10229-g7e035110 (7e035110784fba02ba81944e444be9a36932c6a3) quincy (dev), process ceph-mgr, pid 7 2022-01-18T19:54:52.484+0000 7f8914f04000 0 pidfile_write: ignore empty --pid-file 2022-01-18T19:54:52.510+0000 7f8914f04000 1 mgr[py] Loading python module 'mirroring' 2022-01-18T19:56:14.017+0000 7f890da9f700 -1 failed to decode message of type 87 v2: void pg_stat_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input 2022-01-18T19:56:14.018+0000 7f890d29e700 -1 failed to decode message of type 87 v2: void pg_stat_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input 2022-01-18T19:56:14.018+0000 7f890ca9d700 -1 failed to decode message of type 87 v2: void pg_stat_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input 2022-01-18T19:56:14.019+0000 7f890da9f700 -1 failed to decode message of type 87 v2: void pg_stat_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input 2022-01-18T19:56:14.019+0000 7f890da9f700 -1 failed to decode message of type 87 v2: void pg_stat_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input
[ubuntu@gibba001 ~]$ sudo ceph versions { "mon": { "ceph version 17.0.0-10229-g7e035110 (7e035110784fba02ba81944e444be9a36932c6a3) quincy (dev)": 3, "ceph version 17.0.0-9964-gf2313edc (f2313edc67106699e6ab91f50fa91928e579f7ac) quincy (dev)": 2 }, "mgr": { "ceph version 17.0.0-10229-g7e035110 (7e035110784fba02ba81944e444be9a36932c6a3) quincy (dev)": 2 }, "osd": { "ceph version 17.0.0-9964-gf2313edc (f2313edc67106699e6ab91f50fa91928e579f7ac) quincy (dev)": 975 }, "mds": { "ceph version 17.0.0-9964-gf2313edc (f2313edc67106699e6ab91f50fa91928e579f7ac) quincy (dev)": 2 }, "overall": { "ceph version 17.0.0-10229-g7e035110 (7e035110784fba02ba81944e444be9a36932c6a3) quincy (dev)": 5, "ceph version 17.0.0-9964-gf2313edc (f2313edc67106699e6ab91f50fa91928e579f7ac) quincy (dev)": 979 } }
Updated by Neha Ojha over 2 years ago
- Subject changed from PG Stats unknown after upgrade to 17.0.0-10229-g7e035110 to FAILED to decode MSG_PGSTATS
Updated by Neha Ojha over 2 years ago
- Subject changed from FAILED to decode MSG_PGSTATS to mgr FAILED to decode MSG_PGSTATS
Updated by Vikhyat Umrao over 2 years ago
- Subject changed from mgr FAILED to decode MSG_PGSTATS to [Upgrade] mgr FAILED to decode MSG_PGSTATS
Updated by Neha Ojha over 2 years ago
- Status changed from New to In Progress
- Assignee set to Ronen Friedman
- Priority changed from Normal to Immediate
Updated by Neha Ojha about 2 years ago
- Status changed from In Progress to Can't reproduce
Updated by Aishwarya Mathuria about 2 years ago
- Status changed from Can't reproduce to In Progress
- Assignee changed from Ronen Friedman to Aishwarya Mathuria
Updated by Yuri Weinstein about 2 years ago
Updated by Neha Ojha about 2 years ago
- Status changed from In Progress to Pending Backport
- Backport set to quincy
Updated by Backport Bot about 2 years ago
- Copied to Backport #55137: quincy: [Upgrade] mgr FAILED to decode MSG_PGSTATS added
Updated by Neha Ojha about 2 years ago
- Status changed from Pending Backport to Resolved
Updated by Aishwarya Mathuria about 2 years ago
- Related to Bug #55140: quincy OSD won't start: what(): void pg_stat_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input added
Actions