Project

General

Profile

Bug #38892

/ceph/src/tools/kvstore_tool.cc:266:1: internal compiler error: Segmentation fault

Added by Sebastian Wagner over 1 year ago. Updated 12 months ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
Code Hygiene
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature:

Description

[ 79%] Building CXX object src/tools/CMakeFiles/ceph-kvstore-tool.dir/kvstore_tool.cc.o
/home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/kvstore_tool.cc:266:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
Preprocessed source stored into /tmp/cc51Ovir.out file, please attach this to your bugreport.

This is line 266 this time.


Related issues

Copied from RADOS - Bug #38633: /ceph/src/tools/kvstore_tool.cc:39:1: internal compiler error: Segmentation fault on Bionic Resolved 03/08/2019

History

#1 Updated by Sebastian Wagner over 1 year ago

  • Copied from Bug #38633: /ceph/src/tools/kvstore_tool.cc:39:1: internal compiler error: Segmentation fault on Bionic added

#2 Updated by Sebastian Wagner over 1 year ago

  • Description updated (diff)

#3 Updated by Brad Hubbard over 1 year ago

While I was looking into this I noticed this warning in the Jenkins output.

JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://wiki.jenkins-ci.org/display/JENKINS/Remove+Git+Plugin+BuildsByBranch+BuildData

I wonder whether this is a contributing factor to some of the issues we've seen?

This specific issue appears to be transient as telling Jenkins to try again appears to resolve it in most cases. This may indicate some form of resource contention as the cause.

#4 Updated by Kefu Chai over 1 year ago

per Brad

If we see this again we could try temporarily adding "--param ggc-min-expand=1 --param ggc-min-heapsize=0" to the compiler command line as that seems to prevent the segfault, at least in my testing.

  • ggc-min-expand is the percentage of the gc'heap expansion before reaching which, gc will perform collection. its default value is @std::min(30% + 70% * (RAM/1GB), 100%). changing it to 1 will force gc to try to do gc very often -- it will try to do gc every time its heap size increases by 1%.
  • ggc-min-heapsize is the absolute size of gc before it tries to perform collection. its default value is std::clamp(4MiB, RAM/8, 128MiB), where RAM is std::min_element(prlimit --as --data --rss). 0 will force GCC's gc to collect whenever possible no matter how small the heap is.

#5 Updated by Brad Hubbard over 1 year ago

See https://github.com/ceph/ceph/pull/27479 for a viable workaround. Note that this is a bug in gcc7 [1] and the preferred fix is to move to gcc8 however I wanted to add that PR so people had access to the workaround for this, and any future occurrences.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1531154

#6 Updated by Brad Hubbard over 1 year ago

  • Status changed from New to 15
  • Priority changed from High to Low

#7 Updated by Patrick Donnelly 12 months ago

  • Status changed from 15 to Fix Under Review

#8 Updated by Brad Hubbard 12 months ago

  • Status changed from Fix Under Review to Closed

Also available in: Atom PDF