Bug #52327
closedModule 'devicehealth' has failed: can't subtract offset-naive and offset-aware datetimes mgr/dashboard
0%
Description
Description of problem¶
Since we updated to v16.2.5 (using cephadm since availability) we see this status message :-(
MGR_MODULE_ERROR: Module 'devicehealth' has failed: can't subtract offset-naive and offset-aware datetimes
ceph device get-health-metrics <devid>
also gives a similar message ...
We assume that the failure prediction does also not work anymore too ....
Environment¶
ceph version
string: 16.2.5- Platform (OS/distro/release): Ubuntu 20.04.3 LTS
- Cluster details (nodes, monitors, OSDs): 5 nodes, 5 monitors, 32 OSDs (hdd/ssd)
- Browser used (e.g.:
Version 86.0.4240.198 (Official Build) (64-bit)
):
How reproducible¶
ceph device get-health-metrics <devid>
Actual results¶
Inferring fsid 2af6c250-ccb0-4fe8-80af-f2181098c0e4
Inferring config /var/lib/ceph/2af6c250-ccb0-4fe8-80af-f2181098c0e4/mon.ceph-a/config
Using recent ceph image ceph/ceph@sha256:9b04c0f15704c49591640a37c7adfd40ffad0a4b42fecb950c3407687cb4f29a
Error EIO: Module 'devicehealth' has experienced an error and cannot handle commands: can't subtract offset-naive and offset-aware datetimes
Expected results¶
here
Additional info¶
here
Updated by Harry Coin over 2 years ago
Saw this for the first time just after an upgrade to 16.2.6
Updated by Benno Lange over 2 years ago
Luckily, after the upgrade to v16.2.6 the error seems to be gone :-)
Updated by Harry Coin over 2 years ago
Looks like you are Bilbo and I am Frodo... now I have the ring....
Updated by Josh Durgin over 2 years ago
- Assignee changed from Sage Weil to Yaarit Hatuka
Updated by Yaarit Hatuka over 2 years ago
- Status changed from New to Fix Under Review
- Backport set to pacific, octopus
- Pull request ID set to 43412
Thanks for reporting this, Benno and Harry.
The issue was caused due to a missing timezone in one of the objects.
We have a fix which will be backported.
Updated by Konstantin Shalygin over 2 years ago
Harry Coin wrote:
Still exists in 16.2.7
The fix still under review
Updated by Neha Ojha over 2 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Backport Bot over 2 years ago
- Copied to Backport #53620: octopus: Module 'devicehealth' has failed: can't subtract offset-naive and offset-aware datetimes mgr/dashboard added
Updated by Backport Bot over 2 years ago
- Copied to Backport #53621: pacific: Module 'devicehealth' has failed: can't subtract offset-naive and offset-aware datetimes mgr/dashboard added
Updated by Yaarit Hatuka almost 2 years ago
- Status changed from Pending Backport to Resolved