Feature #40640: Network ping monitoring - RADOS - Ceph

Custom queries

Bug queue
Bug triage
Crash queue
Crash triage
Feedback
My issues
Need Review
Pending backports
Product Backlog Scrub

Actions

Copy link

Feature #40640

closed

Network ping monitoring

Added by David Zafman almost 5 years ago. Updated over 4 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

David Zafman

Category:

Target version:

% Done:

Source:

Tags:

Backport:

luminous, mimic, nautilus

Reviewed:

Affected Versions:

Component(RADOS):

Pull request ID:

28755

Description

The simplest version of this would be to see warnings if heartbeat ping response time exceeds certain thresholds.

Subtasks 1 (1 open — 0 closed)

Feature #41563: Add connection reset tracking to Network ping monitoring

New

David Zafman

07/02/2019

Actions

Related issues 6 (0 open — 6 closed)

Related to RADOS - Bug #41689: Network ping test fails in TEST_network_ping_test2

Resolved

David Zafman

09/06/2019

Actions

Related to RADOS - Bug #41743: Long heartbeat ping times on front interface seen, longest is 2237.999 msec (OSD_SLOW_PING_TIME_FRONT)

Resolved

David Zafman

09/10/2019

Actions

Related to RADOS - Bug #42570: mgr: qa: upgrade mimic-master "src/osd/osd_types.h: 2313: FAILED ceph_assert(pos <= end)"

Resolved

David Zafman

Actions

Copied to RADOS - Backport #41695: nautilus: Network ping monitoring

Resolved

David Zafman

Actions

Copied to RADOS - Backport #41696: mimic: Network ping monitoring

Resolved

David Zafman

Actions

Copied to RADOS - Backport #41697: luminous: Network ping monitoring

Resolved

David Zafman

Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by David Zafman almost 5 years ago

Description updated (diff)

Actions

Copy link

Updated by David Zafman almost 5 years ago

See also https://pad.ceph.com/p/Network_ping_monitoring

Examples, with warning threshold set to 1 microsecond.

Summary status example
SLOW_PING_TIME_BACK Long heartbeat ping times on back interface seen, longest is 1488 msec, SLOW_PING_TIME_FRONT Long heartbeat ping times on front interface seen, longest is 1805 msec

Detail status example
SLOW_PING_TIME_BACK Long heartbeat ping times on back interface seen, longest is 1488 msec
Slow heartbeat ping on back interface from osd.1 to osd.2 1488 msec
Slow heartbeat ping on back interface from osd.0 to osd.2 1412 msec
Slow heartbeat ping on back interface from osd.2 to osd.1 1364 msec
Slow heartbeat ping on back interface from osd.2 to osd.0 1346 msec
Slow heartbeat ping on back interface from osd.0 to osd.1 1310 msec
Truncated long network list. Use ceph daemon osd.# dump_network for more information

SLOW_PING_TIME_FRONT Long heartbeat ping times on front interface seen, longest is 1805 msec
Slow heartbeat ping on front interface from osd.2 to osd.1 1805 msec
Slow heartbeat ping on front interface from osd.0 to osd.2 1648 msec
Slow heartbeat ping on front interface from osd.1 to osd.2 1495 msec
Slow heartbeat ping on front interface from osd.2 to osd.0 1461 msec
Slow heartbeat ping on front interface from osd.1 to osd.0 1448 msec
Truncated long network list. Use ceph daemon osd.# dump_network for more information

Actions

Copy link