Linker segfaults in Jenkins build on Intel and ARM
[ 60%] Building CXX object src/librbd/CMakeFiles/rbd_internal.dir/journal/Replay.cc.o [ 60%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_pubsub_common.cc.o Scanning dependencies of target erasure_code_plugins [ 60%] Built target erasure_code_plugins [ 60%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_realm.cc.o [ 62%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_role.cc.o [ 62%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/rgw_rest_s3.cc.o [ 62%] Building CXX object src/librbd/CMakeFiles/rbd_internal.dir/journal/ResetRequest.cc.o [ 62%] Building CXX object src/crimson/CMakeFiles/crimson-common.dir/__/osd/osd_types.cc.o collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped compilation terminated. src/librados/CMakeFiles/librados.dir/build.make:123: recipe for target 'lib/librados.so.2.0.0' failed make: *** [lib/librados.so.2.0.0] Error 1 make: *** Deleting file 'lib/librados.so.2.0.0' CMakeFiles/Makefile2:4262: recipe for target 'src/librados/CMakeFiles/librados.dir/all' failed make: *** [src/librados/CMakeFiles/librados.dir/all] Error 2 make: *** Waiting for unfinished jobs....
#2 Updated by David Galloway about 1 year ago
- Priority changed from Normal to Urgent
Yuri is seeing this in bionic builds for his wip-yuri6-testing-2019-10-30-1540-nautilus branch as well.
I logged in to a builder this occurred on and see this in the syslog (I think it's related?):
Nov 5 12:36:43 ceph-builders kernel: [ 5684.124493] show_signal_msg: 12 callbacks suppressed Nov 5 12:36:43 ceph-builders kernel: [ 5684.124504] ld: segfault at 7fffd5de9fb8 ip 00007f6d7b9287b5 sp 00007fffd5de9fc0 error 6 in libbfd-2.26.1-system.so[7f6d7b843000+126000]
Adam suggested maybe there's a newer linker version available. Here are the package upgrades available on the builder I checked:
The following packages have been kept back: cpp-5 g++-5 gcc-5 gcc-5-base libasan2 libcilkrts5 libgcc-5-dev libmpx0 libobjc-5-dev libobjc4 libstdc++-5-dev libubsan0 The following packages will be upgraded: apport base-files bind9-host bsdutils btrfs-tools cloud-init dnsutils dpkg friendly-recovery gcc-6-base grub-legacy-ec2 initramfs-tools initramfs-tools-bin initramfs-tools-core libbind9-140 libdns-export162 libdns162 libfdisk1 libisc-export160 libisc160 libisccc140 libisccfg140 liblwres141 libmount1 libpam-systemd libsmartcols1 libsystemd0 mount psmisc python3-apport python3-distupgrade python3-problem-report python3-software-properties python3-update-manager snapd software-properties-common sosreport systemd systemd-sysv ubuntu-core-launcher ubuntu-release-upgrader-core ubuntu-server ubuntu-standard update-manager-core util-linux uuid-runtime 46 upgraded, 0 newly installed, 0 to remove and 12 not upgraded. Need to get 24.2 MB/24.4 MB of archives. After this operation, 1,106 kB of additional disk space will be used.
Do we pin the compiler version somewhere in ceph.git? Can/should we update it? If there's something we should update in ceph-build.git or on the infra side, I'm happy to help. This is outside the scope of my understanding at this point.