Bug #38892
closed/ceph/src/tools/kvstore_tool.cc:266:1: internal compiler error: Segmentation fault
0%
Description
[ 79%] Building CXX object src/tools/CMakeFiles/ceph-kvstore-tool.dir/kvstore_tool.cc.o /home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/kvstore_tool.cc:266:1: internal compiler error: Segmentation fault } ^ Please submit a full bug report, with preprocessed source if appropriate. See <http://bugzilla.redhat.com/bugzilla> for instructions. Preprocessed source stored into /tmp/cc51Ovir.out file, please attach this to your bugreport.
- https://jenkins.ceph.com/job/ceph-pull-requests/16194/
- https://jenkins.ceph.com/job/ceph-pull-requests/16170/consoleFull#14359466752a811ea2-3e7b-466b-84b4-d13df7e35809
- https://jenkins.ceph.com/job/ceph-pull-requests/16173/
This is line 266 this time.
Updated by Sebastian Wagner about 5 years ago
- Copied from Bug #38633: /ceph/src/tools/kvstore_tool.cc:39:1: internal compiler error: Segmentation fault on Bionic added
Updated by Brad Hubbard about 5 years ago
While I was looking into this I noticed this warning in the Jenkins output.
JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://wiki.jenkins-ci.org/display/JENKINS/Remove+Git+Plugin+BuildsByBranch+BuildData
I wonder whether this is a contributing factor to some of the issues we've seen?
This specific issue appears to be transient as telling Jenkins to try again appears to resolve it in most cases. This may indicate some form of resource contention as the cause.
Updated by Kefu Chai about 5 years ago
per Brad
If we see this again we could try temporarily adding "--param ggc-min-expand=1 --param ggc-min-heapsize=0" to the compiler command line as that seems to prevent the segfault, at least in my testing.
- ggc-min-expand is the percentage of the gc'heap expansion before reaching which, gc will perform collection. its default value is @std::min(30% + 70% * (RAM/1GB), 100%). changing it to 1 will force gc to try to do gc very often -- it will try to do gc every time its heap size increases by 1%.
- ggc-min-heapsize is the absolute size of gc before it tries to perform collection. its default value is
std::clamp(4MiB, RAM/8, 128MiB)
, where RAM isstd::min_element(prlimit --as --data --rss)
. 0 will force GCC's gc to collect whenever possible no matter how small the heap is.
Updated by Brad Hubbard about 5 years ago
See https://github.com/ceph/ceph/pull/27479 for a viable workaround. Note that this is a bug in gcc7 [1] and the preferred fix is to move to gcc8 however I wanted to add that PR so people had access to the workaround for this, and any future occurrences.
Updated by Brad Hubbard almost 5 years ago
- Status changed from New to 15
- Priority changed from High to Low
Updated by Patrick Donnelly over 4 years ago
- Status changed from 15 to Fix Under Review
Updated by Brad Hubbard over 4 years ago
- Status changed from Fix Under Review to Closed