Project

General

Profile

Bug #57742

Setting predict_interval in diskprediction_local causes module failure

Added by K Jarrett 4 months ago.

Status:
New
Priority:
Normal
Assignee:
Category:
diskprediction_local
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

After upgrading from 16.2.10 to 17.2.4, a critical health error is raised;

MGR_MODULE_ERROR: Module 'diskprediction_local' has failed: unsupported operand type(s) for %: 'float' and 'str'

After turning on debug logging for this module, the following output is present;

debug 2022-10-01T16:02:52.117+0000 7f9982eb0700  0 [diskprediction_local INFO root] Starting diskprediction local module
debug 2022-10-01T16:02:52.117+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [INF] : Starting diskprediction local module
debug 2022-10-01T16:02:52.117+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  sleep_interval = 600
debug 2022-10-01T16:02:52.117+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  sleep_interval = 600
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  predict_interval = 600
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  predict_interval = 600
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  predictor_model = prophetstor
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  predictor_model = prophetstor
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_level = debug
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_level = debug
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_to_file = False
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_to_file = False
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_to_cluster = True
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_to_cluster = True
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_to_cluster_level = debug
debug 2022-10-01T16:02:52.121+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_to_cluster_level = debug
debug 2022-10-01T16:02:52.125+0000 7f9982eb0700  0 [diskprediction_local DEBUG root] Last predicted 2022-10-01 15:24:38
debug 2022-10-01T16:02:52.125+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] : Last predicted 2022-10-01 15:24:38
debug 2022-10-01T16:02:52.129+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  sleep_interval = 600
debug 2022-10-01T16:02:52.129+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  sleep_interval = 600
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  predict_interval = 600
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  predict_interval = 600
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  predictor_model = prophetstor
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  predictor_model = prophetstor
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_level = debug
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_level = debug
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_to_file = False
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_to_file = False
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_to_cluster = True
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_to_cluster = True
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 [diskprediction_local DEBUG root]  log_to_cluster_level = debug
debug 2022-10-01T16:02:52.133+0000 7f9982eb0700  0 log_channel(diskprediction_local) log [DBG] :  log_to_cluster_level = debug
debug 2022-10-01T16:02:52.141+0000 7f9980e6c700  0 [pg_autoscaler INFO root] _maybe_adjust
debug 2022-10-01T16:02:52.141+0000 7f9982eb0700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'diskprediction_local' while running on mgr.host: unsupported operand type(s) for %: 'float' and 'str'
debug 2022-10-01T16:02:52.141+0000 7f9982eb0700 -1 diskprediction_local.serve:
debug 2022-10-01T16:02:52.141+0000 7f9982eb0700 -1 Traceback (most recent call last):
  File "/usr/share/ceph/mgr/diskprediction_local/module.py", line 92, in serve
    seconds -= seconds % predicted_frequency
TypeError: unsupported operand type(s) for %: 'float' and 'str'

Line 92 currently reads;

seconds -= seconds % predicted_frequency

In the module config, I have explicitly set predict_interval to 600. If I remove this, the module starts up successfully using the default 86400 value instead.

This may have been introduced by this PR which removed the cast to int around predicted_frequency on line 90 without also casting the output from the configuration elsewhere?

Also available in: Atom PDF