Bug #55923: Module 'devicehealth' has failed: unknown operation - cephsqlite - Ceph

Actions

Copy link

Bug #55923

closed

Module 'devicehealth' has failed: unknown operation

Added by Yaarit Hatuka almost 2 years ago. Updated about 1 year ago.

Status:

Closed

Priority:

Normal

Assignee:

Target version:

% Done:

Source:

Development

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Hi Patrick,

Adding this issue for future reference. We saw these crashes in the LRC a few weeks back when upgrading from
17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)
to
17.2.0-176-g6551f450 (6551f450487afe6c3f7bdd7a7c9c891e447045ed) quincy.

2022-05-12T18:29:40.107+0000 7fb9b0e6e700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'devicehealth' while running on mgr.reesi005.xxyjcw: unknown operation
2022-05-12T18:29:40.107+0000 7fb9b0e6e700 -1 devicehealth.serve:
2022-05-12T18:29:40.107+0000 7fb9b0e6e700 -1 Traceback (most recent call last):
  File "/usr/share/ceph/mgr/devicehealth/module.py", line 338, in serve
    if self.db_ready() and self.enable_monitoring:
  File "/usr/share/ceph/mgr/mgr_module.py", line 1189, in db_ready
    return self.db is not None
  File "/usr/share/ceph/mgr/mgr_module.py", line 1201, in db
    self._db = self.open_db()
  File "/usr/share/ceph/mgr/mgr_module.py", line 1182, in open_db
    self.configure_db(db)
  File "/usr/share/ceph/mgr/mgr_module.py", line 1168, in configure_db
    db.execute('PRAGMA JOURNAL_MODE = PERSIST')
sqlite3.InternalError: unknown operation

From the mgr log:

2022-05-12T17:55:25.548+0000 7f48acc66700 -1 Remote method threw exception: Traceback (most recent call last):
File "/usr/share/ceph/mgr/devicehealth/module.py", line 764, in get_recent_device_metrics
return self._get_device_metrics(devid, min_sample=min_sample)
File "/usr/share/ceph/mgr/devicehealth/module.py", line 557, in _get_device_metrics
cursor = self.db.execute(SQL_MIN, (devid, imin_sample))