Bug #15027: Infernalis: backport? mon: pg stuck creating, even though an active+clean pg_stat update was received - Ceph - Ceph

Actions

Copy link

Bug #15027

closed

Infernalis: backport? mon: pg stuck creating, even though an active+clean pg_stat update was received

Added by Sage Weil about 8 years ago. Updated over 7 years ago.

Status:

Can't reproduce

Priority:

Urgent

Assignee:

Sage Weil

Category:

Monitor

Target version:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

2016-03-08 21:49:19.650586 7fb93be13700 15 mon.a@0(leader).pg v1175  got 0.19a reported at 799:6 state creating -> active+clean
...
2016-03-08 21:49:20.404459 7fb93d616700 20 mon.a@0(leader).pg v1175 map_pg_creates  0.19a  acting_primary: 0 -> 4 acting: [0,5,3] -> [4,5,3] up_primary: 4 -> 4 up: [4,5,3] -> [4,5,3]
...
2016-03-08 21:49:20.525075 7fb93d616700 20 mon.a@0(leader).pg v1175  refreshing pg 0.19a got 0 len 557

leaves pg in creating set, and later we see

2016-03-08 22:02:26.866315 7fb93a610700  1 -- 172.21.15.28:6789/0 >> 172.21.15.36:6804/25733 conn(0x7fb95082a800 sd=23 :6789 s=STATE_OPEN pgs=66 cs=1 l=1). == tx == 0x7fb951572900 osd_pg_create(e826 0.19a:793) v3

/a/sage-2016-03-08_12:22:24-rados-wip-sage-testing---basic-smithi/47413

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Sage Weil about 8 years ago

Status changed from New to Need More Info

I take it back.. this wasn't a race. I can't figure out why the pg wasn't removed from the creating_pgs set, though. :(

Actions

Copy link

Updated by Sage Weil about 8 years ago

Subject changed from mon: osdmap update and pg stat creating->active in same commit race to mon: pg stuck creating, even though an active+clean pg_stat update was received

Actions

Copy link

Updated by Sage Weil about 8 years ago

Status changed from Need More Info to Fix Under Review

https://github.com/ceph/ceph/pull/8208

Actions

Copy link

Updated by Sage Weil about 8 years ago

Status changed from Fix Under Review to Resolved

Actions

Copy link

Updated by Samuel Just about 8 years ago

Happened on infernalis. Not sure whether we want to backport. http://tracker.ceph.com/issues/15027

Actions

Copy link

Updated by Samuel Just about 8 years ago

Related to Bug #15550: "AssertionError: failed to recover before timeout expired" in rados-infernalis-distro-basic-openstack added

Actions

Copy link

Updated by Samuel Just about 8 years ago

Subject changed from mon: pg stuck creating, even though an active+clean pg_stat update was received to Infernalis: backport? mon: pg stuck creating, even though an active+clean pg_stat update was received
Status changed from Resolved to 12

['39177', '39119', '39060'] in the same run also

Actions

Copy link