Project

General

Profile

Bug #8720

Calamari: Pg status bar remains clean until refresh.

Added by Warren Usui about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Category:
-
Target version:
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

RHEL 7 -- 1 Calamari Server, 1 Monitor Server, 3 OSDs.

OSD was marked down via the UI. 2 of 3 osds are up according to the monitor.

PG Status is 0/512 Active & Clean

The PG status bar remained green until I got bored (at least 10 minutes)

I hit the refresh button, and the bar changed to yellow.

gmeno - Triage Notes:
There are three events that cause this area of the UI to update.
Numbers of PGs are updated periodically.
The graph seems to be updated by a reactionary event.
It would seem that the bug is that this event is intermittently firing.
The design seems to be intentionally not updating the graph on the 3 second period event.

dashboard_before_refresh (110 KB) Christina Meno, 07/09/2014 11:53 AM

dashboard_after_refresh (110 KB) Christina Meno, 07/09/2014 11:53 AM

health_report (107 KB) Christina Meno, 07/09/2014 11:53 AM

History

#1 Updated by Warren Usui about 7 years ago

Note. When I brought the OSD back up, and PG/Status read 512/512 Active & Clean, I needed to hit the refresh to get the whole bar green again.

#2 Updated by Warren Usui about 7 years ago

This seems to be a problem on RHEL 7. Centos 6.5 works much better. The Bar changes to yellow and back to green without refreshes in the above case. It is also mixed red and green as I reboot sites, and when all the sites are back up it is green again (all with no screen refreshes)

#3 Updated by Warren Usui about 7 years ago

  • Assignee set to Christina Meno

#4 Updated by Warren Usui about 7 years ago

Note: This happens on both Chrome and Firefox

#5 Updated by Warren Usui about 7 years ago

Note: this problem seems to only happen when osds go down. Pg status is 0/xxx Active & Clean, but the status bar does not change until refresh. In all other tests (osd's going up, pg number changes, reboots...) the bar seems to update by itself.

#6 Updated by Christina Meno about 7 years ago

Steps to reproduce:

in one tab load dashboard
in another tab goto workbench take an osd out.
look at first tab with dashboard

see screenshots

#7 Updated by Warren Usui about 7 years ago

I have now seen this on RHel 6.5. Here's what I did.

I have 448/448 pg active and clean. The bar is all green

I mark an OSD out. The pg status shows 0/448. The status bar remains green.
I refresh the page. The status bar is yellow.

I mark the OSD in. The pg status grows back to 448/448. The status bar remains yellow.
I refresh the screen. The status bar is green.

#8 Updated by Warren Usui about 7 years ago

I should catch up on email before I update issues.

#9 Updated by Christina Meno about 7 years ago

  • Target version set to v1.3-dev1

#10 Updated by Warren Usui about 7 years ago

This bug seems to happen on Firefox. It does not happen on Chrome (although I have done thirty times as many tests on Firefox than I have on Chrome),

#11 Updated by Christina Meno about 7 years ago

  • Target version changed from v1.3-dev1 to 1.3-dev2

#12 Updated by Christina Meno about 7 years ago

  • Description updated (diff)

#13 Updated by Christina Meno about 7 years ago

  • Status changed from New to In Progress

#14 Updated by Christina Meno about 7 years ago

  • Status changed from In Progress to Fix Under Review

I was wrong about the signal not being propagated.
It now seems that countPGs() is subtly wrong.

I hacked this by updating the pgmap if count changed or not.
I measured the impact to performance and even when drawing a full graph it still only takes N ms time.

https://github.com/ceph/calamari-clients/pull/43

#15 Updated by Christina Meno about 7 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF