Project

General

Profile

Feature #35687

rgw: storing and reading total usage data to construct rgw service monitor by prometheus

Added by Oshyn Song about 1 year ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
09/05/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Component(RADOS):
Pull request ID:

Description

There are problems for the current rgw usage data storing and reading implementation:

1. The usage data will be accumulating to a large amount by time. If we delete the history usage data by the "trim usage" API, we will lost the history data permanently if the usage data of the rgw user or bucket is critical for the application(such as billing of each API).
2. Current rgw usage data is storing in a record and accumulated by each hour, we can not get more details data or even real-time usage data to construct the monitoring system by prometheus and grafana.
3. We can also use the current implementation to construct the monitoring facility by sum all each-hour data, but it will generate high overhead if the history data grows too large.

If we store the total usage data in a record, and support reading it by the "read usage" admin OP API, the monitoring system will be easily built with prometheus and grafana. I have implement the `radosgw_exporter`(https://github.com/oshynsong/radosgw_exporter), and it depends on the rgw "read usage" admin OP API.

Also available in: Atom PDF