Project

General

Profile

Actions

Bug #20095

closed

remove/reduce master executors

Added by Alfredo Deza almost 7 years ago. Updated almost 6 years ago.

Status:
Closed
Priority:
High
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

At a point where almost all the 20 executors where being used, top showed the Java process consuming lots of RAM and CPU.

This is causing the process to stall and Nginx will return a 504.


Files

top.png (42.6 KB) top.png Alfredo Deza, 05/26/2017 03:37 PM
Actions #1

Updated by Alfredo Deza almost 7 years ago

  • Description updated (diff)
Actions #2

Updated by Alfredo Deza almost 7 years ago

Actions #3

Updated by Alfredo Deza almost 7 years ago

  • File deleted (Screen Shot 2017-05-26 at 11.30.59 AM.png)
Actions #4

Updated by Alfredo Deza almost 7 years ago

This also caused Jenkins to choke on 300 commits pushed to ceph/master within minutes of each other.

Jenkins cannot have 20 executors, we need to be able to scale this to mitigate a bottleneck on the master instance.

Actions #5

Updated by David Galloway almost 6 years ago

  • Status changed from New to Closed
  • Assignee set to David Galloway

I feel this has been mitigated by creating the ability to have multiple Jenkins masters.

Actions

Also available in: Atom PDF