Project

General

Profile

Actions

Bug #50491

closed

mgr/dashboard: centralized logging

Added by Ernesto Puerta about 3 years ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Category:
Monitoring
Target version:
% Done:

100%

Source:
Tags:
backport_processed
Backport:
quincy pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

User Story

As a Ceph operator I want to have a unified view of the logs from the different daemons, so that:
  • I can perform a backward/post-mortem analysis of events leading to an issue,
  • I can monitor cluster events in real-time.

Persona

  • Ceph cluster operator/sys admin
  • Support engineer
  • Developers

Context

Every daemon in Ceph stores its logs locally (there's a "cluster log" but it's extremely concise, and not useful for troubleshooting). This means that if a user wants to perform a post-mortem analysis of an issue, they have first to collect log traces from multiple hosts, which involves to:
  1. Identify on which host a daemon is running,
  2. Log in to that host,
  3. Look for the log file in the filesystem,
  4. Open the log and perform a search.

For debugging a Ceph issue, users often have to follow the operational events from multiple daemons, so this tasks gets more and more complicated. Additionally, it's almost impossible to perform real-time (vs. post-mortem) troubleshooting.

Implementation details

To explore multiple stacks: ELK, Fluentd, Loki, etc.

This might be embedded via iframe as already done for Grafana dashboards, or accessed stand-alone.

References

SUSE's Ceph + ELK


Related issues 2 (0 open2 closed)

Copied to Dashboard - Backport #54582: pacific: mgr/dashboard: centralized loggingRejectedAashish SharmaActions
Copied to Dashboard - Backport #54583: quincy: mgr/dashboard: centralized loggingResolvedActions
Actions

Also available in: Atom PDF