Project

General

Profile

Bug #42596

Linker segfaults in Jenkins build on Intel and ARM

Added by Adam Emerson about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

[ 60%] Building CXX object src/librbd/CMakeFiles/rbd_internal.dir/journal/Replay.cc.o
[ 60%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_pubsub_common.cc.o
Scanning dependencies of target erasure_code_plugins
[ 60%] Built target erasure_code_plugins
[ 60%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_realm.cc.o
[ 62%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_role.cc.o
[ 62%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_s3.cc.o
[ 62%] Building CXX object src/librbd/CMakeFiles/rbd_internal.dir/journal/ResetRequest.cc.o
[ 62%] Building CXX object src/crimson/CMakeFiles/crimson-common.dir/__/osd/osd_types.cc.o
collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
compilation terminated.
src/librados/CMakeFiles/librados.dir/build.make:123: recipe for target 'lib/librados.so.2.0.0' failed
make[3]: *** [lib/librados.so.2.0.0] Error 1
make[3]: *** Deleting file 'lib/librados.so.2.0.0'
CMakeFiles/Makefile2:4262: recipe for target 'src/librados/CMakeFiles/librados.dir/all' failed
make[2]: *** [src/librados/CMakeFiles/librados.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....

From https://jenkins.ceph.com/job/ceph-pull-requests/37673/consoleFull#-853622487be8fa57d-c354-45fc-9e5c-e33905c76575

History

#1 Updated by Adam Emerson about 1 year ago

  • Subject changed from Linker segfaults in Jenkins build on Intel to Linker segfaults in Jenkins build on Intel and ARM
  • Severity changed from 3 - minor to 2 - major

#2 Updated by David Galloway about 1 year ago

  • Priority changed from Normal to Urgent

Yuri is seeing this in bionic builds for his wip-yuri6-testing-2019-10-30-1540-nautilus branch as well.

https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=bionic,DIST=bionic,MACHINE_SIZE=huge/32422/

I logged in to a builder this occurred on and see this in the syslog (I think it's related?):

Nov  5 12:36:43 ceph-builders kernel: [ 5684.124493] show_signal_msg: 12 callbacks suppressed
Nov  5 12:36:43 ceph-builders kernel: [ 5684.124504] ld[10184]: segfault at 7fffd5de9fb8 ip 00007f6d7b9287b5 sp 00007fffd5de9fc0 error 6 in libbfd-2.26.1-system.so[7f6d7b843000+126000]

Adam suggested maybe there's a newer linker version available. Here are the package upgrades available on the builder I checked:

The following packages have been kept back:
  cpp-5 g++-5 gcc-5 gcc-5-base libasan2 libcilkrts5 libgcc-5-dev libmpx0 libobjc-5-dev libobjc4 libstdc++-5-dev libubsan0
The following packages will be upgraded:
  apport base-files bind9-host bsdutils btrfs-tools cloud-init dnsutils dpkg friendly-recovery gcc-6-base grub-legacy-ec2 initramfs-tools initramfs-tools-bin initramfs-tools-core libbind9-140 libdns-export162 libdns162 libfdisk1 libisc-export160 libisc160 libisccc140
  libisccfg140 liblwres141 libmount1 libpam-systemd libsmartcols1 libsystemd0 mount psmisc python3-apport python3-distupgrade python3-problem-report python3-software-properties python3-update-manager snapd software-properties-common sosreport systemd systemd-sysv
  ubuntu-core-launcher ubuntu-release-upgrader-core ubuntu-server ubuntu-standard update-manager-core util-linux uuid-runtime
46 upgraded, 0 newly installed, 0 to remove and 12 not upgraded.
Need to get 24.2 MB/24.4 MB of archives.
After this operation, 1,106 kB of additional disk space will be used.

Do we pin the compiler version somewhere in ceph.git? Can/should we update it? If there's something we should update in ceph-build.git or on the infra side, I'm happy to help. This is outside the scope of my understanding at this point.

#4 Updated by Adam Emerson about 1 year ago

  • Project changed from rgw to Ceph

#5 Updated by Kefu Chai about 1 year ago

  • Status changed from New to Fix Under Review
  • Assignee set to Kefu Chai
  • Pull request ID set to 31601

#6 Updated by Kefu Chai about 1 year ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF