Bug #21518: mgr[zabbix] float division by zero - mgr - Ceph

Actions

Copy link

Bug #21518

closed

mgr[zabbix] float division by zero

Added by Ilja Slepnev over 6 years ago. Updated over 6 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Category:

ceph-mgr

Target version:

% Done:

Source:

Community (user)

Tags:

Backport:

luminous

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v12.2.0

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Enabled Zabbix module, configured as described on http://docs.ceph.com/docs/master/mgr/zabbix/, but no data is received.
float division by zero is logged in ceph-mgr.log

ceph-mgr.log ceph-mgr.log

2017-09-22 22:30:05.867985 7fcb03ae9700 20 mgr[zabbix] zabbix_port: 10051
2017-09-22 22:30:05.868022 7fcb03ae9700 20 mgr[zabbix] zabbix_host: 10.18.10.18
2017-09-22 22:30:05.868056 7fcb03ae9700 20 mgr[zabbix] identifier: lhep-serv-3
2017-09-22 22:30:05.868091 7fcb03ae9700 20 mgr[zabbix] zabbix_sender: /usr/bin/zabbix_sender
2017-09-22 22:30:05.868134 7fcb03ae9700 20 mgr[zabbix] interval: 60
2017-09-22 22:30:05.868171 7fcb03ae9700 20 mgr[zabbix] Waking up for new iteration
2017-09-22 22:30:05.868924 7fcb03ae9700 20 mgr[zabbix] Sending data to Zabbix server 10.18.10.18
2017-09-22 22:30:05.868988 7fcb03ae9700 20 mgr[zabbix] {'osd_full_ratio': 0.949999988079071, 'total_objects': 0L, 'num_mon': 3, 'rd_ops': 0L, 'num_pg_temp': 9, 'total_used_bytes': 0L, 'num_osd_up': 6, 'wr_ops': 0L, 'rd_bytes': 0L, 'osd_nearfull_ratio': 0.8500000238418579, 'num_osd_in': 5, 'num_pools': 1, 'overall_status': u'HEALTH_WARN', 'osd_backfillfull_ratio': 0.8999999761581421, 'total_bytes': 0L, 'num_pg': 64L, 'overall_status_int': 1, 'wr_bytes': 0L, 'num_osd': 6, 'total_avail_bytes': 0L}
2017-09-22 22:30:05.878460 7fcb03ae9700  0 mgr[zabbix] Exception when sending: /usr/bin/zabbix_sender exited non-zero: zabbix_sender [1121]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 20; total: 20; seconds spent: 0.000120"}]
2017-09-22 22:30:05.878626 7fcb03ae9700 20 mgr[zabbix] Sleeping for 60 seconds

2017-09-22 22:31:05.878997 7fcb03ae9700 20 mgr[zabbix] Waking up for new iteration
2017-09-22 22:31:05.879650 7fcb03ae9700  0 mgr[zabbix] float division by zero
2017-09-22 22:31:05.879723 7fcb03ae9700 20 mgr[zabbix] Sleeping for 60 seconds

2017-09-22 22:32:05.880052 7fcb03ae9700 20 mgr[zabbix] Waking up for new iteration
2017-09-22 22:32:05.880937 7fcb03ae9700  0 mgr[zabbix] float division by zero
2017-09-22 22:32:05.881018 7fcb03ae9700 20 mgr[zabbix] Sleeping for 60 seconds

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by Ilja Slepnev over 6 years ago

After upgrading all OSDs to bluestore, the issue resolved.

One of OSD had filestore format while others had bluestore. This may be related to the issue.
No more "division by zero" message and data is actually sent by zabbix_sender, verified by using tcpdump.

ceph-mgr.log ceph-mgr.log

2017-09-22 23:04:05.957635 7fcb03ae9700 20 mgr[zabbix] Waking up for new iteration
2017-09-22 23:04:05.958547 7fcb03ae9700 20 mgr[zabbix] Sending data to Zabbix server 10.18.10.18
2017-09-22 23:04:05.958625 7fcb03ae9700 20 mgr[zabbix] {'rd_bytes': 11398143492096L, 'total_bytes': 4798027112448L, 'overall_status_int': 1, 'osd_latency_apply_min': 0L, 'osd_latency_apply_max': 5L, 'num_osd_up': 6, 'osd_max_fill': 13.095749553280234, 'osd_backfillfull_ratio': 0.8999999761581421, 'osd_latency_commit_max': 5L, 'wr_bytes': 3337205972992L, 'osd_latency_commit_min': 0L, 'num_mon': 3, 'osd_min_fill': 2.037110421878282, 'total_used_bytes': 359502389248L, 'wr_ops': 305991965L, 'osd_nearfull_ratio': 0.8500000238418579, 'osd_latency_apply_avg': 1.0, 'overall_status': u'HEALTH_WARN', 'num_pg': 64L, 'osd_latency_commit_avg': 1.0, 'osd_avg_fill': 7.492748589967889, 'num_osd': 6, 'osd_full_ratio': 0.949999988079071, 'total_objects': 28475L, 'num_pools': 1, 'num_osd_in': 6, 'num_pg_temp': 24, 'rd_ops': 181606152L, 'total_avail_bytes': 4438524723200L}
2017-09-22 23:04:05.965294 7fcb03ae9700  0 mgr[zabbix] Exception when sending: /usr/bin/zabbix_sender exited non-zero: zabbix_sender [9297]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 29; total: 29; seconds spent: 0.000180"}]
2017-09-22 23:04:05.965396 7fcb03ae9700 20 mgr[zabbix] Sleeping for 60 seconds

Actions

Copy link