Project

General

Profile

Actions
Copy link

Bug #50591

closed

mgr/progress: progress can be negative

Added by Dan van der Ster almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
Dan van der Ster
Category:
progress module
Target version:
Ceph - v17.0.0
% Done:

40%

Source:
Community (user)
Tags:
Backport:
nautilus,octopus,pacific
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
Ceph - v14.2.19
ceph-qa-suite:
Pull request ID:
41094
Crash signature (v1):
Crash signature (v2):

Description

Seen in #50587: progress can be negative. This causes unresponsive mons in nautilus <= v14.2.20 when responding to `ceph status`.

2021-04-29 17:39:42.340 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577965..3577966
2021-04-29 17:40:42.423 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577966..3577967
2021-04-29 17:40:42.589 7fbfe2b47700  1 mgr[progress] osd.321 marked out
2021-04-29 17:40:43.072 7fbfe2b47700  1 mgr[progress] 0 PGs affected by osd.321 being marked out
2021-04-29 17:40:43.521 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577967..3577968
2021-04-29 17:40:44.669 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577968..3577969
2021-04-29 17:40:45.769 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577969..3577970
2021-04-29 17:40:46.879 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577970..3577971
2021-04-29 17:40:48.010 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577971..3577972
2021-04-29 17:40:56.083 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577972..3577973
2021-04-29 17:41:12.081 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577973..3577974
2021-04-29 17:41:12.489 7fbfe2b47700  1 mgr[progress] osd.321 marked in
2021-04-29 17:41:12.974 7fbfe2b47700  1 mgr[progress] 3 PGs affected by osd.321 being marked in
2021-04-29 17:41:18.097 7fbfe2b47700  4 mgr[progress] Updated progress to 0.0 (Rebalancing after osd.321 marked in)
2021-04-29 17:41:18.684 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577974..3577976
2021-04-29 17:41:24.592 7fbfe2b47700  4 mgr[progress] Updated progress to 0.0 (Rebalancing after osd.321 marked in)
2021-04-29 17:41:30.448 7fbfe2b47700  4 mgr[progress] Updated progress to -0.333333333333 (Rebalancing after osd.321 marked in)
2021-04-29 17:41:30.722 7fbfe2b47700  4 mgr[progress] Processing OSDMap change 3577976..3577977
2021-04-29 17:41:31.222 7fbfe2b47700  1 mgr[progress] osd.538 marked in
2021-04-29 17:41:31.721 7fbfe2b47700  1 mgr[progress] 0 PGs affected by osd.538 being marked in
2021-04-29 17:41:36.768 7fbfe2b47700  4 mgr[progress] Updated progress to -0.333333333333 (Rebalancing after osd.321 marked in)
2021-04-29 17:41:42.117 7fbfe2b47700  4 mgr[progress] Updated progress to -0.333333333333 (Rebalancing after osd.321 marked in)
2021-04-29 17:41:47.358 7fbfe2b47700  4 mgr[progress] Updated progress to -0.333333333333 (Rebalancing after osd.321 marked in)
2021-04-29 17:41:52.587 7fbfe2b47700  4 mgr[progress] Updated progress to -0.333333333333 (Rebalancing after osd.321 marked in)

Related issues 4 (0 open — 4 closed)

Copied from RADOS - Bug #50587: mon election storm following osd recreation: huge tcmalloc and ceph::msgr::v2::FrameAssembler backtracesResolved

Actions
Copied to mgr - Backport #50777: nautilus: mgr/progress: progress can be negativeResolvedKonstantin ShalyginActions
Copied to mgr - Backport #50778: octopus: mgr/progress: progress can be negativeResolvedKonstantin ShalyginActions
Copied to mgr - Backport #50779: pacific: mgr/progress: progress can be negativeResolvedKonstantin ShalyginActions
Actions
Copy link
 #1

Updated by Dan van der Ster almost 3 years ago

  • Copied from Bug #50587: mon election storm following osd recreation: huge tcmalloc and ceph::msgr::v2::FrameAssembler backtraces added
Actions
Copy link
 #2

Updated by Dan van der Ster almost 3 years ago

  • Status changed from New to Fix Under Review
  • Assignee set to Dan van der Ster
  • Backport set to nautilus,octopus,pacific
Actions
Copy link
 #3

Updated by Dan van der Ster almost 3 years ago

  • Pull request ID set to 41094
Actions
Copy link
 #4

Updated by Konstantin Shalygin almost 3 years ago

  • Category set to progress module
  • % Done changed from 0 to 40
  • Source set to Community (user)
Actions
Copy link
 #5

Updated by Kefu Chai almost 3 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #6

Updated by Backport Bot almost 3 years ago

  • Copied to Backport #50777: nautilus: mgr/progress: progress can be negative added
Actions
Copy link
 #7

Updated by Backport Bot almost 3 years ago

  • Copied to Backport #50778: octopus: mgr/progress: progress can be negative added
Actions
Copy link
 #8

Updated by Backport Bot almost 3 years ago

  • Copied to Backport #50779: pacific: mgr/progress: progress can be negative added
Actions
Copy link
 #9

Updated by Loïc Dachary almost 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions
Copy link

Also available in: Atom PDF